2024-03-08T05:49:12.647 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-08T05:49:12.647 DEBUG:teuthology.run:Teuthology command: teuthology --name jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi --owner scheduled_jcollin@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin} --verbose --archive /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425 -- /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425/orig.config.yaml 2024-03-08T05:49:12.690 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-08T05:49:12.774 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425 branch: main description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin} email: jcollin@redhat.com first_in_suite: false job_id: '7585425' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: lockdep: true ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - missing required features sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 kclient: syntax: v1 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-jcollin-test_per-client-perf-counters sha1: de7a7e179fbee9e314fa702f8f6c09d121c5e1a7 owner: scheduled_jcollin@teuthology priority: 10 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7313 sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-jcollin-test_per-client-perf-counters suite_path: /home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa suite_relpath: qa suite_repo: https://github.com/joscollin/ceph.git suite_sha1: de7a7e179fbee9e314fa702f8f6c09d121c5e1a7 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE31JNxLJ5xIvqhIrwCXCY/HPljTsB8D0lzrE06ebM6bqC9FPeEn+hA8Jfmm1hUfxwY8fvklHKl+ayl8QGGNwVM= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgUVmyKe/dO23KvZNq7LUj18qPW9uMLNPo7GkfiWAQj+cwmNKxO+iImtt3ST8tLKKwIeAz0qcdYZJTcyo+gcoQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_admin.TestLabeledPerfCounters teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-08_05:43:46 tube: smithi user: jcollin verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-08T05:49:12.774 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa; will attempt to use it 2024-03-08T05:49:12.775 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa/tasks 2024-03-08T05:49:12.775 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-08T05:49:12.777 INFO:teuthology.task.internal:Checking packages... 2024-03-08T05:49:12.797 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1fb5d10608dccee73edd6ef8a2c7b804a097c324' 2024-03-08T05:49:12.798 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-08T05:49:12.798 INFO:teuthology.packaging:ref: None 2024-03-08T05:49:12.798 INFO:teuthology.packaging:tag: None 2024-03-08T05:49:12.798 INFO:teuthology.packaging:branch: main 2024-03-08T05:49:12.798 INFO:teuthology.packaging:sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T05:49:12.798 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-03-08T05:49:13.055 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1940.g639d1827 2024-03-08T05:49:13.056 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-08T05:49:13.065 INFO:teuthology.task.internal:no buildpackages task found 2024-03-08T05:49:13.066 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-08T05:49:13.122 INFO:teuthology.task.internal:Saving configuration 2024-03-08T05:49:13.137 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-08T05:49:13.146 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T05:49:13.170 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 05:44:38.759492', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE31JNxLJ5xIvqhIrwCXCY/HPljTsB8D0lzrE06ebM6bqC9FPeEn+hA8Jfmm1hUfxwY8fvklHKl+ayl8QGGNwVM='} 2024-03-08T05:49:13.193 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 05:44:38.761226', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgUVmyKe/dO23KvZNq7LUj18qPW9uMLNPo7GkfiWAQj+cwmNKxO+iImtt3ST8tLKKwIeAz0qcdYZJTcyo+gcoQ='} 2024-03-08T05:49:13.194 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-08T05:49:13.200 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-08T05:49:13.200 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-08T05:49:13.200 INFO:teuthology.run_tasks:Running task console_log... 2024-03-08T05:49:13.290 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2c242c70d0>, signals=[15]) 2024-03-08T05:49:13.291 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-08T05:49:13.298 INFO:teuthology.task.internal:Opening connections... 2024-03-08T05:49:13.298 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T05:49:13.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:49:13.405 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T05:49:13.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:49:13.509 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-08T05:49:13.516 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-03-08T05:49:13.534 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-03-08T05:49:13.534 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T05:49:13.592 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T05:49:13.593 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-03-08T05:49:13.616 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-03-08T05:49:13.634 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-03-08T05:49:13.634 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-03-08T05:49:13.691 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-03-08T05:49:13.691 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T05:49:13.692 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T05:49:13.693 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-03-08T05:49:13.707 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-08T05:49:13.717 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-08T05:49:13.738 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-08T05:49:13.739 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-03-08T05:49:13.741 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-03-08T05:49:13.758 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-08T05:49:13.764 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-08T05:49:13.764 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-03-08T05:49:13.798 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-03-08T05:49:13.820 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-08T05:49:13.873 INFO:teuthology.run_tasks:Running task kernel... 2024-03-08T05:49:13.887 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-08T05:49:13.888 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-03-08T05:49:13.888 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-03-08T05:49:13.888 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T05:49:13.889 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T05:49:13.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T05:49:13.905 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-08T05:49:13.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T05:49:13.909 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-08T05:49:14.134 DEBUG:teuthology.task.kernel:sha1 for client.0 is dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T05:49:14.134 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412"... 2024-03-08T05:49:14.134 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-03-08T05:49:14.146 DEBUG:teuthology.task.kernel:sha1 for client.1 is dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T05:49:14.146 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412"... 2024-03-08T05:49:14.146 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-03-08T05:49:14.152 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-361.el9.x86_64 2024-03-08T05:49:14.152 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T05:49:14.152 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-08T05:49:14.153 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum upgrade -y linux-firmware 2024-03-08T05:49:14.164 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-361.el9.x86_64 2024-03-08T05:49:14.164 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T05:49:14.164 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-08T05:49:14.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum upgrade -y linux-firmware 2024-03-08T05:49:16.363 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-08T05:49:17.947 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - BaseOS 4.6 MB/s | 2.0 MB 00:00 2024-03-08T05:49:18.041 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-03-08T05:49:18.940 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-03-08T05:49:20.046 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-03-08T05:49:21.466 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-03-08T05:49:22.113 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-03-08T05:49:26.575 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.1 MB 00:01 2024-03-08T05:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.1 MB 00:00 2024-03-08T05:49:28.140 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 15 kB 00:00 2024-03-08T05:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 15 kB 00:00 2024-03-08T05:49:29.755 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-08T05:49:34.885 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-03-08T05:49:37.288 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 1 Package 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 375 M 2024-03-08T05:49:37.304 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-03-08T05:49:47.528 INFO:teuthology.orchestra.run.smithi040.stdout:linux-firmware-20240219-143.el9.noarch.rpm 37 MB/s | 375 MB 00:10 2024-03-08T05:49:47.529 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-03-08T05:49:47.529 INFO:teuthology.orchestra.run.smithi040.stdout:Total 37 MB/s | 375 MB 00:10 2024-03-08T05:49:47.529 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-03-08T05:49:47.612 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-03-08T05:49:47.612 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-03-08T05:49:50.098 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-03-08T05:49:50.098 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-03-08T05:49:56.692 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-03-08T05:49:56.976 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-08T05:49:57.072 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T05:49:58.368 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T05:49:58.368 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-08T05:49:58.762 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T05:49:58.762 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T05:49:58.762 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-03-08T05:49:58.762 INFO:teuthology.orchestra.run.smithi040.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-08T05:49:58.762 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T05:49:58.762 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-03-08T05:49:58.930 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 686 kB/s | 21 MB 00:30 2024-03-08T05:49:59.045 INFO:teuthology.task.kernel:Downloading kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.0... 2024-03-08T05:49:59.045 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T05:49:59.188 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ 2024-03-08T05:49:59.188 DEBUG:teuthology.orchestra.run.smithi040:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-08T05:49:59.190 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi040.front.sepia.ceph.com to complete... 2024-03-08T05:50:04.039 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-08T05:50:06.460 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-03-08T05:50:06.474 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-03-08T05:50:06.474 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-03-08T05:50:06.474 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-03-08T05:50:06.474 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-03-08T05:50:06.475 INFO:teuthology.orchestra.run.smithi053.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-08T05:50:06.475 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T05:50:06.475 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-03-08T05:50:06.475 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-03-08T05:50:06.475 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 1 Package 2024-03-08T05:50:06.475 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T05:50:06.475 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 375 M 2024-03-08T05:50:06.475 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-03-08T05:50:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08 05:50:12 URL:https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm [336216379/336216379] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-08T05:50:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:FINISHED --2024-03-08 05:50:12-- 2024-03-08T05:50:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:Total wall clock time: 13s 2024-03-08T05:50:12.332 INFO:teuthology.orchestra.run.smithi040.stderr:Downloaded: 1 files, 321M in 13s (25.0 MB/s) 2024-03-08T05:50:12.335 INFO:teuthology.task.kernel:Installing kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.0... 2024-03-08T05:50:12.335 DEBUG:teuthology.orchestra.run.smithi040:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-08T05:50:14.374 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying... ######################################## 2024-03-08T05:50:14.429 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing... ######################################## 2024-03-08T05:50:14.515 INFO:teuthology.orchestra.run.smithi040.stdout:Updating / installing... 2024-03-08T05:50:22.429 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-6.8.0_rc7_gdd5c1cd17167-1 ######################################## 2024-03-08T05:50:24.321 INFO:teuthology.orchestra.run.smithi053.stdout:linux-firmware-20240219-143.el9.noarch.rpm 21 MB/s | 375 MB 00:17 2024-03-08T05:50:24.322 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-03-08T05:50:24.322 INFO:teuthology.orchestra.run.smithi053.stdout:Total 21 MB/s | 375 MB 00:17 2024-03-08T05:50:24.323 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-03-08T05:50:24.417 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-03-08T05:50:24.417 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-03-08T05:50:26.901 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-03-08T05:50:26.901 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-03-08T05:50:33.427 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-03-08T05:50:33.653 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-08T05:50:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T05:50:34.852 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T05:50:34.852 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-08T05:50:35.258 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T05:50:35.259 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T05:50:35.259 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-03-08T05:50:35.259 INFO:teuthology.orchestra.run.smithi053.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-08T05:50:35.259 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T05:50:35.259 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-03-08T05:50:35.534 INFO:teuthology.task.kernel:Downloading kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.1... 2024-03-08T05:50:35.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T05:50:35.677 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ 2024-03-08T05:50:35.678 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-08T05:50:35.680 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi053.front.sepia.ceph.com to complete... 2024-03-08T05:50:38.831 INFO:teuthology.orchestra.run.smithi040.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-08T05:51:01.994 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi040.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-08T05:51:01.994 DEBUG:teuthology.orchestra.run.smithi040:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/System.map 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/config 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-08T05:51:02.081 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel 2024-03-08T05:51:02.082 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm.ko 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ansi_cprng.ko 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/anubis.ko 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/arc4.ko 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_memcpy.ko 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_pq.ko 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_tx.ko 2024-03-08T05:51:02.083 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_xor.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/raid6test.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authenc.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authencesn.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blake2b_generic.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_common.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_generic.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/camellia_generic.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast5_generic.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast6_generic.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast_common.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ccm.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cmac.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crc32_generic.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_engine.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_user.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cts.ko 2024-03-08T05:51:02.084 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/des_generic.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/dh_generic.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/echainiv.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/essiv.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/fcrypt.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/khazad.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4hc.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/md4.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/michael_mic.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcbc.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcrypt.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/rmd160.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/seed.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/serpent_generic.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tcrypt.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tea.ko 2024-03-08T05:51:02.085 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_common.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_generic.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/vmac.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/wp512.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xcbc.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xor.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xxhash_generic.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_pad.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/custom_method.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/ec_sys.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/platform_profile.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbs.ko 2024-03-08T05:51:02.086 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbshc.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/video.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/acard-ahci.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ahci_platform.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ata_generic.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/libahci_platform.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_acpi.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ali.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_amd.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_artop.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atiixp.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atp867x.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd640.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd64x.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cypress.ko 2024-03-08T05:51:02.087 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_efar.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt366.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt37x.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it8213.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it821x.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_jmicron.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_marvell.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_mpiix.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_netcell.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ninja32.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87410.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87415.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_oldpiix.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_opti.ko 2024-03-08T05:51:02.088 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_optidma.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pcmcia.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_piccolo.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_rdc.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sch.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_serverworks.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sil680.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sis.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sl82c105.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_triflex.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_via.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pdc_adma.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_inic162x.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_mv.ko 2024-03-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_nv.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_promise.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_qstor.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil24.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sis.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_svw.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sx4.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_uli.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_via.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_vsc.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay/ks0108.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-08T05:51:02.090 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma/bcma.ko 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe/aoe.ko 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/brd.ko 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd/drbd.ko 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/floppy.ko 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/loop.ko 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/nbd.ko 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk/null_blk.ko 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/pktcdvd.ko 2024-03-08T05:51:02.091 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/rbd.ko 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/virtio_blk.ko 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkfront.ko 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram/zram.ko 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hangcheck-timer.ko 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/via-rng.ko 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-08T05:51:02.092 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/lp.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave/mwave.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ppdev.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tlclk.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-08T05:51:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/uv_mmtimer.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/virtio_console.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp.ko 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel 2024-03-08T05:51:02.094 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-aes.ko 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-sha.ko 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca/dca.ko 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-08T05:51:02.095 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/amd64_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/e752x_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/edac_mce_amd.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3000_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3200_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5100_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5400_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7300_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7core_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i82975x_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/ie31200_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/sb_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/x38_edac.ko 2024-03-08T05:51:02.096 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/edd.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-ich.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-appleir.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-aureal.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-axff.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-dr.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elecom.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elo.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-emsff.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gaff.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gt683r.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gyration.ko 2024-03-08T05:51:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtekff.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-hyperv.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-icade.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-keytouch.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-kye.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lcpower.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-led.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lenovo.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-multitouch.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-ortek.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-penmount.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-petalynx.ko 2024-03-08T05:51:02.098 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-picolcd.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-pl.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-plantronics.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-primax.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-rmi.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-common.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-08T05:51:02.099 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-saitek.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-samsung.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sjoy.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sony.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-speedlink.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-steelseries.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sunplus.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tivo.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tmff.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-topseed.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-twinhan.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-uclogic.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-waltop.ko 2024-03-08T05:51:02.100 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-wiimote.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-xinmo.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zpff.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zydacron.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/uhid.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/wacom.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_balloon.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_utils.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_vmbus.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru3.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7414.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7418.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adc128d818.ko 2024-03-08T05:51:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1025.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1026.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1029.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1031.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm9240.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ads7828.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7410.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7411.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7462.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7470.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7475.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7x10.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/amc6821.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/applesmc.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asb100.ko 2024-03-08T05:51:02.102 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asc7621.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/atxp1.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/coretemp.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dme1737.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds1621.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds620.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc1403.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc6w201.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71805f.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71882fg.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f75375s.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fam15h_power.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fschmd.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g760a.ko 2024-03-08T05:51:02.103 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g762.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl518sm.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl520sm.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5500_temp.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5k_amb.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmaem.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmpex.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina209.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina2xx.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/it87.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k10temp.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k8temp.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lineage-pem.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm63.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm73.ko 2024-03-08T05:51:02.104 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm75.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm77.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm78.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm80.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm83.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm85.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm87.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm90.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm92.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm93.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95234.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95241.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95245.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc2945.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4151.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4215.ko 2024-03-08T05:51:02.105 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4222.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4245.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4260.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4261.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max16065.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1619.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1668.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max197.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6639.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6650.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6697.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/mcp3021.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6683.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775-core.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct7802.ko 2024-03-08T05:51:02.106 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87360.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87427.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pcf8591.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-08T05:51:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/powr1220.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5627.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5636.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht15.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht21.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/shtc1.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sis5595.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47b397.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m1.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m192.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/thmc50.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp102.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp103.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp401.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp421.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via-cputemp.ko 2024-03-08T05:51:02.108 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via686a.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt1211.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt8231.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627ehf.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627hf.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83781d.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83791d.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83792d.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83793.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83795.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l785ts.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l786ng.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-08T05:51:02.109 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-08T05:51:02.110 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-dev.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-mux.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-smbus.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-stub.ko 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core 2024-03-08T05:51:02.111 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_core.ko 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-08T05:51:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/gameport.ko 2024-03-08T05:51:02.113 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/joydev.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/appletouch.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/bcm5974.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/cyapatp.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/sermouse.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/altera_ps2.ko 2024-03-08T05:51:02.114 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/arc_ps2.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/serio_raw.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/sparse-keymap.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-blinkm.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lm3530.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lp3944.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-ss4200.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-08T05:51:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache/bcache.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-bio-prison.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache-smq.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-crypt.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-delay.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-flakey.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-log-userspace.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-multipath.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-queue-length.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-raid.ko 2024-03-08T05:51:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-round-robin.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-service-time.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-switch.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-thin-pool.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-verity.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid0.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid1.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid10.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid456.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptbase.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptctl.ko 2024-03-08T05:51:02.117 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptfc.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptlan.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptsas.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptscsih.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptspi.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_ich.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_sch.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/mfd-core.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/sm501.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/viperboard.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/vx855.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/wl1273-core.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-08T05:51:02.118 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds9802als.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds990x.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/bh1770glc.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710/cb710.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/at24.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/max6875.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/enclosure.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/hpilo.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29003.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29020.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-08T05:51:02.119 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-me.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-txe.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_7xx1.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_core.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tsl2550.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_balloon.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-08T05:51:02.120 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding/bonding.ko 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dummy.ko 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/eql.ko 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-08T05:51:02.121 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-08T05:51:02.122 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-08T05:51:02.123 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-08T05:51:02.124 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-08T05:51:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dnet.ko 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ethoc.ko 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/fealnx.ko 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e 2024-03-08T05:51:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/jme.ko 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-08T05:51:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-08T05:51:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-08T05:51:02.129 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc 2024-03-08T05:51:02.130 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-08T05:51:02.131 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/geneve.ko 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ifb.ko 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-08T05:51:02.132 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvlan.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvtap.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mii.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/net_failover.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/netconsole.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/nlmon.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/amd.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-08T05:51:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm87xx.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/broadcom.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/cicada.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/davicom.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/dp83640.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/et1011c.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/icplus.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/lxt.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/marvell.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/micrel.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/national.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/phylink.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/qsemi.ko 2024-03-08T05:51:02.134 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/realtek.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/smsc.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/ste10Xp.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/vitesse.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_async.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppoe.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppox.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pptp.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slhc.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slip.ko 2024-03-08T05:51:02.135 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/sungem_phy.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tap.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_random.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tun.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/veth.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/virtio_net.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan/vxlan.ko 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback 2024-03-08T05:51:02.136 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netfront.ko 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb/ntb.ko 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme-core.ko 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme.ko 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport.ko 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_cs.ko 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_pc.ko 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_serial.ko 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-08T05:51:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/xen-pcifront.ko 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/i82092.ko 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/pd6729.ko 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acerhdf.ko 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-08T05:51:02.138 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/hdaps.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel_ips.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-08T05:51:02.139 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps_parport.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/pps_core.ko 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp 2024-03-08T05:51:02.140 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp_kvm.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-em3027.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-08T05:51:02.141 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-max6900.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-08T05:51:02.142 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-x1205.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-9xxx.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-sas.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-xxxx.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/BusLogic.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/a100u2w.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/advansys.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/am53c974.ko 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr 2024-03-08T05:51:02.143 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/atp870u.ko 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa/bfa.ko 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ch.ko 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-08T05:51:02.144 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dc395x.ko 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dmx3191d.ko 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esp_scsi.ko 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-08T05:51:02.145 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic/fnic.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hpsa.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hptiop.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hv_storvsc.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/initio.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ipr.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ips.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci/isci.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc/libfc.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi.ko 2024-03-08T05:51:02.146 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas/libsas.ko 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid.ko 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvumi.ko 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-08T05:51:02.147 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pmcraid.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla1280.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/raid_class.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_debug.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ses.ko 2024-03-08T05:51:02.148 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/st.ko 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/stex.ko 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/virtio_scsi.ko 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/wd719x.ko 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb/ssb.ko 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_file.ko 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_iblock.ko 2024-03-08T05:51:02.149 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_mod.ko 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_pscsi.ko 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_user.ko 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-08T05:51:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_gsm.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_hdlc.ko 2024-03-08T05:51:02.151 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/nozomi.ko 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/arc_uart.ko 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio.ko 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_aec.ko 2024-03-08T05:51:02.152 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_cif.ko 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_pci_generic.ko 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_sercos3.ko 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-acm.ko 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usblp.ko 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usbtmc.ko 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/mdc800.ko 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/microtek.ko 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc 2024-03-08T05:51:02.153 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/adutux.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/appledisplay.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi26.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi62.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ezusb.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/idmouse.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/iowarrior.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ldusb.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/legousbtower.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usb3503.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usblcd.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-08T05:51:02.154 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/uss720.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/yurex.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy/phy-generic.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/aircable.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ark3116.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ch341.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cp210x.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cyberjack.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/empeg.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-08T05:51:02.155 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_ti.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipaq.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipw.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ir-usb.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mct_u232.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7720.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7840.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/navman.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/omninet.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/opticon.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/option.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/oti6858.ko 2024-03-08T05:51:02.156 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/pl2303.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcaux.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcserial.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/quatech2.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/safe_serial.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/sierra.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ssu100.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/symbolserial.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_debug.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/visor.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/whiteheat.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage 2024-03-08T05:51:02.157 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/uas.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-karma.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/usb-storage.ko 2024-03-08T05:51:02.158 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio.ko 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost.ko 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_net.ko 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_scsi.ko 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video 2024-03-08T05:51:02.159 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/apple_bl.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/lcd.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_balloon.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_mmio.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_ring.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2482.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2490.ko 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves 2024-03-08T05:51:02.160 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/wire.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-08T05:51:02.161 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/hpwdt.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/i6300esb.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ib700wdt.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ibmasr.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it87_wdt.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/machzwd.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/nv_tco.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/softdog.ko 2024-03-08T05:51:02.162 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/via_wdt.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/wdt_pci.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/xen_wdt.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-evtchn.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntalloc.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntdev.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-privcmd.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-08T05:51:02.163 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p/9p.ko 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs/affs.ko 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs/befs.ko 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/binfmt_misc.ko 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs/btrfs.ko 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles/cachefiles.ko 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph/ceph.ko 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda/coda.ko 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs 2024-03-08T05:51:02.164 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs/cramfs.ko 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm/dlm.ko 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs/f2fs.ko 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/fat.ko 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/msdos.ko 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/vfat.ko 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/cuse.ko 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/fuse.ko 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2 2024-03-08T05:51:02.165 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2/gfs2.ko 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs/hfs.ko 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus/hfsplus.ko 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs/isofs.ko 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs/jfs.ko 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd/lockd.ko 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix/minix.ko 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs/netfs.ko 2024-03-08T05:51:02.166 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfs.ko 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv3.ko 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv4.ko 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/grace.ko 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/nfs_acl.ko 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd/nfsd.ko 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2/nilfs2.ko 2024-03-08T05:51:02.167 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-celtic.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-centeuro.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-croatian.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-cyrillic.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-gaelic.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-greek.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-iceland.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-inuit.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-roman.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-romanian.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-turkish.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1250.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1251.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1255.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp737.ko 2024-03-08T05:51:02.168 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp775.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp850.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp852.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp855.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp857.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp860.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp861.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp862.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp863.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp864.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp865.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp866.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp869.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp874.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp932.ko 2024-03-08T05:51:02.169 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp936.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp949.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp950.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_euc-jp.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-1.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-13.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-14.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-15.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-2.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-3.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-4.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-5.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-6.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-7.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-9.ko 2024-03-08T05:51:02.170 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-r.ko 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-ru.ko 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-u.ko 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_utf8.ko 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2.ko 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-08T05:51:02.171 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs/overlay.ko 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore/ramoops.ko 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs/reiserfs.ko 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs/romfs.ko 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client/cifs.ko 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_arc4.ko 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_md4.ko 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs 2024-03-08T05:51:02.172 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs/squashfs.ko 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv/sysv.ko 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf/udf.ko 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs/ufs.ko 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs/xfs.ko 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu/rcutorture.ko 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/torture.ko 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-08T05:51:02.173 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/asn1_encoder.ko 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc-itu-t.ko 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc8.ko 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libarc4.ko 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libdes.ko 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/libcrc32c.ko 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lru_cache.ko 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4_compress.ko 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4hc_compress.ko 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math/cordic.ko 2024-03-08T05:51:02.174 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6/raid6_pq.ko 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_bm.ko 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_fsm.ko 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_kmp.ko 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd/zstd_compress.ko 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm/hwpoison-inject.ko 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/6lowpan.ko 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_dest.ko 2024-03-08T05:51:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_fragment.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_hop.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_mobility.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_routing.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_udp.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/garp.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/mrp.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/p8022.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/psnap.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/stp.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q/8021q.ko 2024-03-08T05:51:02.176 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet.ko 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_fd.ko 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_rdma.ko 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_virtio.ko 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv/batman-adv.ko 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/br_netfilter.ko 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/bridge.ko 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-08T05:51:02.177 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-08T05:51:02.178 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtables.ko 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph/libceph.ko 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/failover.ko 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/pktgen.ko 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp 2024-03-08T05:51:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp.ko 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_diag.ko 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv4.ko 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv6.ko 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver/dns_resolver.ko 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/dsa_core.ko 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_brcm.ko 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_dsa.ko 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_none.ko 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_trailer.ko 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan 2024-03-08T05:51:02.180 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ah4.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/esp4.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/fou.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/gre.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/inet_diag.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_gre.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_tunnel.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_vti.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipcomp.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipip.ko 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter 2024-03-08T05:51:02.181 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-08T05:51:02.182 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_bic.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_dctcp.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_diag.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_highspeed.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_htcp.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_hybla.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_illinois.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_lp.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_scalable.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_vegas.ko 2024-03-08T05:51:02.183 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_veno.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_westwood.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_yeah.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tunnel4.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_diag.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_tunnel.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ah6.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/esp6.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/fou6.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_tunnel.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_vti.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ipcomp6.ko 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter 2024-03-08T05:51:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-08T05:51:02.185 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/sit.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/tunnel6.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key/af_key.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_core.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-08T05:51:02.186 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_eth.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip6.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_netlink.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ppp.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc/llc.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154/mac802154.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls/mpls_gso.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-08T05:51:02.187 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-08T05:51:02.188 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conncount.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-08T05:51:02.189 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_log_syslog.ko 2024-03-08T05:51:02.190 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_irc.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_sip.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_tables.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_log.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_chain_nat.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_compat.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_ct.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-08T05:51:02.191 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fib.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_hash.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_limit.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_log.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_masq.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_nat.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_numgen.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_queue.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_quota.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_redir.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject_inet.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_socket.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_tunnel.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_AUDIT.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-08T05:51:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CT.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_DSCP.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HL.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HMARK.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LED.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LOG.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NETMAP.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFLOG.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_RATEEST.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_SECMARK.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-08T05:51:02.193 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TEE.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TPROXY.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TRACE.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_addrtype.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_bpf.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cgroup.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cluster.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_comment.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connbytes.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlabel.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlimit.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connmark.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_conntrack.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cpu.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dccp.ko 2024-03-08T05:51:02.194 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_devgroup.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dscp.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ecn.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_esp.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hashlimit.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_helper.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hl.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipcomp.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_iprange.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipvs.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_l2tp.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_length.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_limit.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mac.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mark.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_multiport.ko 2024-03-08T05:51:02.195 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nat.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nfacct.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_osf.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_owner.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_physdev.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_pkttype.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_policy.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_quota.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_rateest.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_realm.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_recent.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_sctp.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_set.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_socket.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_state.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_statistic.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_string.ko 2024-03-08T05:51:02.196 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_tcpmss.ko 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_time.ko 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_u32.ko 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink/netlink_diag.ko 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh/nsh.ko 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/openvswitch.ko 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-geneve.ko 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-gre.ko 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-vxlan.ko 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet/af_packet_diag.ko 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds.ko 2024-03-08T05:51:02.197 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_rdma.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_tcp.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_csum.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_gact.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_mirred.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_nat.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_pedit.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_police.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_simple.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_skbedit.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_vlan.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_basic.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_bpf.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_flow.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_fw.ko 2024-03-08T05:51:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_route.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_u32.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_cmp.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_ipset.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_meta.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_nbyte.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_text.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_u32.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_choke.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_codel.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_drr.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_fq.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_gred.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hfsc.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hhf.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_htb.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_ingress.ko 2024-03-08T05:51:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio_lib.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_multiq.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_netem.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_pie.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_plug.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_prio.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_qfq.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_red.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfb.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfq.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_tbf.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_teql.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp_diag.ko 2024-03-08T05:51:02.200 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/sunrpc.ko 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/diag.ko 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/tipc.ko 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix/unix_diag.ko 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock.ko 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys/trusted.ko 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib/irqbypass.ko 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias.bin 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin 2024-03-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.alias.bin 2024-03-08T05:51:02.203 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.bin 2024-03-08T05:51:02.203 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.modinfo 2024-03-08T05:51:02.203 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep 2024-03-08T05:51:02.203 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep.bin 2024-03-08T05:51:02.203 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.devname 2024-03-08T05:51:02.203 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.order 2024-03-08T05:51:02.203 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.softdep 2024-03-08T05:51:02.203 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols 2024-03-08T05:51:02.203 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols.bin 2024-03-08T05:51:02.203 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-08T05:51:02.207 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:51:02.207 DEBUG:teuthology.orchestra.run.smithi040:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-08T05:51:02.230 INFO:teuthology.orchestra.run.smithi040.stdout:postinstall scriptlet (using /bin/sh): 2024-03-08T05:51:02.230 INFO:teuthology.orchestra.run.smithi040.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-08T05:51:02.230 INFO:teuthology.orchestra.run.smithi040.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gdd5c1cd17167 /lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-08T05:51:02.230 INFO:teuthology.orchestra.run.smithi040.stdout:fi 2024-03-08T05:51:02.230 INFO:teuthology.orchestra.run.smithi040.stdout:for file in vmlinuz System.map config; do 2024-03-08T05:51:02.230 INFO:teuthology.orchestra.run.smithi040.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167"; then 2024-03-08T05:51:02.230 INFO:teuthology.orchestra.run.smithi040.stdout: cp "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167" 2024-03-08T05:51:02.230 INFO:teuthology.orchestra.run.smithi040.stdout: fi 2024-03-08T05:51:02.230 INFO:teuthology.orchestra.run.smithi040.stdout:done 2024-03-08T05:51:02.230 INFO:teuthology.orchestra.run.smithi040.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-08T05:51:02.230 INFO:teuthology.orchestra.run.smithi040.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-08T05:51:02.231 INFO:teuthology.orchestra.run.smithi040.stdout:new-kernel-pkg --remove 6.8.0-rc7-gdd5c1cd17167 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gdd5c1cd17167.img 2024-03-08T05:51:02.231 INFO:teuthology.orchestra.run.smithi040.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-08T05:51:02.231 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-install remove 6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:51:02.231 INFO:teuthology.orchestra.run.smithi040.stdout:fi 2024-03-08T05:51:02.231 INFO:teuthology.orchestra.run.smithi040.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-08T05:51:02.231 INFO:teuthology.orchestra.run.smithi040.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-08T05:51:02.231 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:51:02.231 INFO:teuthology.orchestra.run.smithi040.stdout:fi 2024-03-08T05:51:02.232 DEBUG:teuthology.orchestra.run.smithi040:> sudo rpm -qi grub2-tools 2024-03-08T05:51:02.321 INFO:teuthology.orchestra.run.smithi040.stdout:Name : grub2-tools 2024-03-08T05:51:02.321 INFO:teuthology.orchestra.run.smithi040.stdout:Epoch : 1 2024-03-08T05:51:02.321 INFO:teuthology.orchestra.run.smithi040.stdout:Version : 2.06 2024-03-08T05:51:02.321 INFO:teuthology.orchestra.run.smithi040.stdout:Release : 68.el9 2024-03-08T05:51:02.321 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: x86_64 2024-03-08T05:51:02.321 INFO:teuthology.orchestra.run.smithi040.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:Group : Unspecified 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:Size : 8264254 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:License : GPLv3+ 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:Packager : builder@centos.org 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:Vendor : CentOS 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:Summary : Support tools for GRUB. 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout:Description : 2024-03-08T05:51:02.322 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T05:51:02.323 INFO:teuthology.orchestra.run.smithi040.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-08T05:51:02.323 INFO:teuthology.orchestra.run.smithi040.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-08T05:51:02.323 INFO:teuthology.orchestra.run.smithi040.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-08T05:51:02.323 INFO:teuthology.orchestra.run.smithi040.stdout:hardware devices. 2024-03-08T05:51:02.323 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T05:51:02.323 INFO:teuthology.orchestra.run.smithi040.stdout:This subpackage provides tools for support of all platforms. 2024-03-08T05:51:02.324 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-08T05:51:02.324 INFO:teuthology.task.kernel:Updating grub on smithi040 to boot 6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:51:02.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-08T05:51:02.825 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2024-03-08T05:51:04.321 INFO:teuthology.orchestra.run.smithi040.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-08T05:51:04.349 INFO:teuthology.orchestra.run.smithi040.stderr:done 2024-03-08T05:51:04.364 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-03-08T05:51:04.379 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.blrV1hPWEn 2024-03-08T05:51:04.379 DEBUG:teuthology.orchestra.run.smithi040:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.blrV1hPWEn 2024-03-08T05:51:04.450 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.blrV1hPWEn 2024-03-08T05:51:04.591 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.blrV1hPWEn is 6KB 2024-03-08T05:51:04.640 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.blrV1hPWEn 2024-03-08T05:51:04.655 DEBUG:teuthology.orchestra.run.smithi040:> sudo /bin/ls /boot/loader/entries || true 2024-03-08T05:51:04.723 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-08T05:51:04.723 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-08T05:51:04.723 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-08T05:51:04.723 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167.conf 2024-03-08T05:51:04.724 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:51:04.881 DEBUG:teuthology.orchestra.run.smithi040:> sudo shutdown -r now 2024-03-08T05:51:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08 05:51:11 URL:https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm [336216379/336216379] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-08T05:51:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:FINISHED --2024-03-08 05:51:11-- 2024-03-08T05:51:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:Total wall clock time: 36s 2024-03-08T05:51:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:Downloaded: 1 files, 321M in 35s (9.12 MB/s) 2024-03-08T05:51:11.357 INFO:teuthology.task.kernel:Installing kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.1... 2024-03-08T05:51:11.357 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-08T05:51:13.446 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying... ######################################## 2024-03-08T05:51:13.507 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing... ######################################## 2024-03-08T05:51:13.591 INFO:teuthology.orchestra.run.smithi053.stdout:Updating / installing... 2024-03-08T05:51:21.428 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-6.8.0_rc7_gdd5c1cd17167-1 ######################################## 2024-03-08T05:51:34.884 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-08T05:51:34.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-03-08T05:51:34.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:51:35.697 INFO:teuthology.orchestra.run.smithi053.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-08T05:51:53.375 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.40 2024-03-08T05:51:59.087 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-08T05:51:59.087 DEBUG:teuthology.orchestra.run.smithi053:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-08T05:51:59.202 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:51:59.202 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/System.map 2024-03-08T05:51:59.202 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/config 2024-03-08T05:51:59.202 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel 2024-03-08T05:51:59.202 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch 2024-03-08T05:51:59.202 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-08T05:51:59.203 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm.ko 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto 2024-03-08T05:51:59.204 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ansi_cprng.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/anubis.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/arc4.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_memcpy.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_pq.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_tx.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_xor.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/raid6test.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authenc.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authencesn.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blake2b_generic.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_common.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_generic.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/camellia_generic.ko 2024-03-08T05:51:59.205 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast5_generic.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast6_generic.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast_common.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ccm.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cmac.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crc32_generic.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_engine.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_user.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cts.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/des_generic.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/dh_generic.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/echainiv.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/essiv.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/fcrypt.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/khazad.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4.ko 2024-03-08T05:51:59.206 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4hc.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/md4.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/michael_mic.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcbc.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcrypt.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/rmd160.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/seed.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/serpent_generic.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tcrypt.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tea.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_common.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_generic.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/vmac.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/wp512.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xcbc.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xor.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xxhash_generic.ko 2024-03-08T05:51:59.207 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_pad.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/custom_method.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/ec_sys.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/platform_profile.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbs.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbshc.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/video.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/acard-ahci.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ahci_platform.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ata_generic.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/libahci_platform.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_acpi.ko 2024-03-08T05:51:59.208 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ali.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_amd.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_artop.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atiixp.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atp867x.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd640.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd64x.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cypress.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_efar.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt366.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt37x.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it8213.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it821x.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_jmicron.ko 2024-03-08T05:51:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_marvell.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_mpiix.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_netcell.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ninja32.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87410.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87415.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_oldpiix.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_opti.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_optidma.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pcmcia.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_piccolo.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_rdc.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sch.ko 2024-03-08T05:51:59.210 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_serverworks.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sil680.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sis.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sl82c105.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_triflex.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_via.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pdc_adma.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_inic162x.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_mv.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_nv.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_promise.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_qstor.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil24.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sis.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_svw.ko 2024-03-08T05:51:59.211 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sx4.ko 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_uli.ko 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_via.ko 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_vsc.ko 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay/ks0108.ko 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma/bcma.ko 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe/aoe.ko 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/brd.ko 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd 2024-03-08T05:51:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd/drbd.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/floppy.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/loop.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/nbd.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk/null_blk.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/pktcdvd.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/rbd.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/virtio_blk.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkfront.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram/zram.ko 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char 2024-03-08T05:51:59.213 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hangcheck-timer.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/via-rng.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/lp.ko 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave 2024-03-08T05:51:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave/mwave.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ppdev.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tlclk.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/uv_mmtimer.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/virtio_console.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-08T05:51:59.215 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp.ko 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-aes.ko 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-sha.ko 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca 2024-03-08T05:51:59.216 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca/dca.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/amd64_edac.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/e752x_edac.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/edac_mce_amd.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3000_edac.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3200_edac.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5100_edac.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5400_edac.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7300_edac.ko 2024-03-08T05:51:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7core_edac.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i82975x_edac.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/ie31200_edac.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/sb_edac.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/x38_edac.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/edd.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-ich.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-appleir.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-aureal.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-axff.ko 2024-03-08T05:51:59.218 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-dr.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elecom.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elo.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-emsff.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gaff.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gt683r.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gyration.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtekff.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-hyperv.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-icade.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-keytouch.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-kye.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lcpower.ko 2024-03-08T05:51:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-led.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lenovo.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-multitouch.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-ortek.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-penmount.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-petalynx.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-picolcd.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-pl.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-plantronics.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-primax.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-rmi.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-common.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-08T05:51:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-saitek.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-samsung.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sjoy.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sony.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-speedlink.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-steelseries.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sunplus.ko 2024-03-08T05:51:59.221 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tivo.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tmff.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-topseed.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-twinhan.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-uclogic.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-waltop.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-wiimote.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-xinmo.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zpff.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zydacron.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/uhid.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/wacom.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_balloon.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_utils.ko 2024-03-08T05:51:59.222 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_vmbus.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru3.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7414.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7418.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adc128d818.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1025.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1026.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1029.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1031.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm9240.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ads7828.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7410.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7411.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7462.ko 2024-03-08T05:51:59.223 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7470.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7475.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7x10.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/amc6821.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/applesmc.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asb100.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asc7621.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/atxp1.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/coretemp.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dme1737.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds1621.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds620.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc1403.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc6w201.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71805f.ko 2024-03-08T05:51:59.224 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71882fg.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f75375s.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fam15h_power.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fschmd.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g760a.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g762.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl518sm.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl520sm.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5500_temp.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5k_amb.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmaem.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmpex.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina209.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina2xx.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/it87.ko 2024-03-08T05:51:59.225 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k10temp.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k8temp.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lineage-pem.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm63.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm73.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm75.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm77.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm78.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm80.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm83.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm85.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm87.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm90.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm92.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm93.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95234.ko 2024-03-08T05:51:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95241.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95245.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc2945.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4151.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4215.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4222.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4245.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4260.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4261.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max16065.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1619.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1668.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max197.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6639.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6650.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6697.ko 2024-03-08T05:51:59.227 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/mcp3021.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6683.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775-core.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct7802.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87360.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87427.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pcf8591.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-08T05:51:59.228 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/powr1220.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5627.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5636.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht15.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht21.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/shtc1.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sis5595.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47b397.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m1.ko 2024-03-08T05:51:59.229 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m192.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/thmc50.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp102.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp103.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp401.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp421.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via-cputemp.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via686a.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt1211.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt8231.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627ehf.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627hf.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83781d.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83791d.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83792d.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83793.ko 2024-03-08T05:51:59.230 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83795.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l785ts.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l786ng.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-08T05:51:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-dev.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-mux.ko 2024-03-08T05:51:59.232 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-smbus.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-stub.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_core.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-08T05:51:59.233 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input 2024-03-08T05:51:59.234 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/gameport.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/joydev.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/appletouch.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/bcm5974.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/cyapatp.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/sermouse.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4 2024-03-08T05:51:59.235 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/altera_ps2.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/arc_ps2.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/serio_raw.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/sparse-keymap.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-blinkm.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lm3530.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lp3944.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-ss4200.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-08T05:51:59.236 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache/bcache.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-bio-prison.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache-smq.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-crypt.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-delay.ko 2024-03-08T05:51:59.237 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-flakey.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-log-userspace.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-multipath.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-queue-length.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-raid.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-round-robin.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-service-time.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-switch.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-thin-pool.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-verity.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid0.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid1.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid10.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid456.ko 2024-03-08T05:51:59.238 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptbase.ko 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptctl.ko 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptfc.ko 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptlan.ko 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptsas.ko 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptscsih.ko 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptspi.ko 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_ich.ko 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_sch.ko 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/mfd-core.ko 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/sm501.ko 2024-03-08T05:51:59.239 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/viperboard.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/vx855.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/wl1273-core.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds9802als.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds990x.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/bh1770glc.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710/cb710.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/at24.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/max6875.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/enclosure.ko 2024-03-08T05:51:59.240 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/hpilo.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29003.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29020.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-me.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-txe.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_7xx1.ko 2024-03-08T05:51:59.241 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_core.ko 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tsl2550.ko 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_balloon.ko 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding/bonding.ko 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx 2024-03-08T05:51:59.242 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dummy.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/eql.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-08T05:51:59.243 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-08T05:51:59.244 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-08T05:51:59.245 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic 2024-03-08T05:51:59.246 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dnet.ko 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex 2024-03-08T05:51:59.247 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ethoc.ko 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/fealnx.ko 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-08T05:51:59.248 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/jme.ko 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-08T05:51:59.249 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-08T05:51:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker 2024-03-08T05:51:59.251 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-08T05:51:59.252 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom 2024-03-08T05:51:59.253 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/geneve.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ifb.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvlan.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvtap.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mii.ko 2024-03-08T05:51:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/net_failover.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/netconsole.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/nlmon.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/amd.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm87xx.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/broadcom.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/cicada.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/davicom.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/dp83640.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/et1011c.ko 2024-03-08T05:51:59.255 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/icplus.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/lxt.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/marvell.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/micrel.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/national.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/phylink.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/qsemi.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/realtek.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/smsc.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/ste10Xp.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/vitesse.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_async.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-08T05:51:59.256 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppoe.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppox.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pptp.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slhc.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slip.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/sungem_phy.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tap.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_random.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-08T05:51:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tun.ko 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/veth.ko 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/virtio_net.ko 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan/vxlan.ko 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netfront.ko 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb/ntb.ko 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme-core.ko 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme.ko 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport 2024-03-08T05:51:59.258 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport.ko 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_cs.ko 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_pc.ko 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_serial.ko 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/xen-pcifront.ko 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/i82092.ko 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/pd6729.ko 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl 2024-03-08T05:51:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acerhdf.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/hdaps.ko 2024-03-08T05:51:59.260 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel_ips.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-08T05:51:59.261 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps_parport.ko 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/pps_core.ko 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp.ko 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp_kvm.ko 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-08T05:51:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-em3027.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-max6900.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-08T05:51:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-x1205.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-9xxx.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-sas.ko 2024-03-08T05:51:59.264 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-xxxx.ko 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/BusLogic.ko 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/a100u2w.ko 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/advansys.ko 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/am53c974.ko 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/atp870u.ko 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa 2024-03-08T05:51:59.265 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa/bfa.ko 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ch.ko 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-08T05:51:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dc395x.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dmx3191d.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esp_scsi.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic/fnic.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hpsa.ko 2024-03-08T05:51:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hptiop.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hv_storvsc.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/initio.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ipr.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ips.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci/isci.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc/libfc.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas/libsas.ko 2024-03-08T05:51:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid.ko 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvumi.ko 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-08T05:51:59.269 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pmcraid.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla1280.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/raid_class.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_debug.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-08T05:51:59.270 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ses.ko 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/st.ko 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/stex.ko 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/virtio_scsi.ko 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/wd719x.ko 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb/ssb.ko 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-08T05:51:59.271 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_file.ko 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_iblock.ko 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_mod.ko 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_pscsi.ko 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_user.ko 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-08T05:51:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless 2024-03-08T05:51:59.273 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_gsm.ko 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_hdlc.ko 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/nozomi.ko 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/arc_uart.ko 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-08T05:51:59.274 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio.ko 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_aec.ko 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_cif.ko 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_pci_generic.ko 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_sercos3.ko 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-acm.ko 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usblp.ko 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usbtmc.ko 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/mdc800.ko 2024-03-08T05:51:59.275 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/microtek.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/adutux.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/appledisplay.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi26.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi62.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ezusb.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/idmouse.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/iowarrior.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ldusb.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/legousbtower.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usb3503.ko 2024-03-08T05:51:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usblcd.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/uss720.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/yurex.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy/phy-generic.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/aircable.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ark3116.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ch341.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cp210x.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cyberjack.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/empeg.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-08T05:51:59.277 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_ti.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipaq.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipw.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ir-usb.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mct_u232.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7720.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7840.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/navman.ko 2024-03-08T05:51:59.278 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/omninet.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/opticon.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/option.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/oti6858.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/pl2303.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcaux.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcserial.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/quatech2.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/safe_serial.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/sierra.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ssu100.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/symbolserial.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_debug.ko 2024-03-08T05:51:59.279 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/visor.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/whiteheat.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/uas.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-karma.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-08T05:51:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/usb-storage.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost 2024-03-08T05:51:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_net.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_scsi.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/apple_bl.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/lcd.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_balloon.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_mmio.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-08T05:51:59.282 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_ring.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2482.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2490.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-08T05:51:59.283 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/wire.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/hpwdt.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/i6300esb.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ib700wdt.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ibmasr.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it87_wdt.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/machzwd.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/nv_tco.ko 2024-03-08T05:51:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/softdog.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/via_wdt.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/wdt_pci.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/xen_wdt.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-08T05:51:59.285 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-evtchn.ko 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntalloc.ko 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntdev.ko 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-privcmd.ko 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p/9p.ko 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs/affs.ko 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs 2024-03-08T05:51:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs/befs.ko 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/binfmt_misc.ko 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs/btrfs.ko 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles/cachefiles.ko 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph/ceph.ko 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda/coda.ko 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs/cramfs.ko 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm/dlm.ko 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-08T05:51:59.287 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs/f2fs.ko 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/fat.ko 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/msdos.ko 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/vfat.ko 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/cuse.ko 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/fuse.ko 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2/gfs2.ko 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs/hfs.ko 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus/hfsplus.ko 2024-03-08T05:51:59.288 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs/isofs.ko 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs/jfs.ko 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd/lockd.ko 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix/minix.ko 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs/netfs.ko 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout 2024-03-08T05:51:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfs.ko 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv3.ko 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv4.ko 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/grace.ko 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/nfs_acl.ko 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd/nfsd.ko 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2/nilfs2.ko 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-celtic.ko 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-centeuro.ko 2024-03-08T05:51:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-croatian.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-cyrillic.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-gaelic.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-greek.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-iceland.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-inuit.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-roman.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-romanian.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-turkish.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1250.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1251.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1255.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp737.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp775.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp850.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp852.ko 2024-03-08T05:51:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp855.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp857.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp860.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp861.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp862.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp863.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp864.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp865.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp866.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp869.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp874.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp932.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp936.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp949.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp950.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_euc-jp.ko 2024-03-08T05:51:59.292 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-1.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-13.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-14.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-15.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-2.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-3.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-4.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-5.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-6.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-7.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-9.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-r.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-ru.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-u.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_utf8.ko 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2 2024-03-08T05:51:59.293 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2.ko 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs/overlay.ko 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore/ramoops.ko 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs/reiserfs.ko 2024-03-08T05:51:59.294 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs/romfs.ko 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client/cifs.ko 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_arc4.ko 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_md4.ko 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs/squashfs.ko 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv/sysv.ko 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf/udf.ko 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs/ufs.ko 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs 2024-03-08T05:51:59.295 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs/xfs.ko 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu/rcutorture.ko 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/torture.ko 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/asn1_encoder.ko 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc-itu-t.ko 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc8.ko 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libarc4.ko 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libdes.ko 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/libcrc32c.ko 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lru_cache.ko 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4 2024-03-08T05:51:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4_compress.ko 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4hc_compress.ko 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math/cordic.ko 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6/raid6_pq.ko 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_bm.ko 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_fsm.ko 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_kmp.ko 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd/zstd_compress.ko 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm/hwpoison-inject.ko 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net 2024-03-08T05:51:59.297 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/6lowpan.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_dest.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_fragment.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_hop.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_mobility.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_routing.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_udp.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/garp.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/mrp.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/p8022.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/psnap.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/stp.ko 2024-03-08T05:51:59.298 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q/8021q.ko 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet.ko 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_fd.ko 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_rdma.ko 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_virtio.ko 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv/batman-adv.ko 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/br_netfilter.ko 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/bridge.ko 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-08T05:51:59.299 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-08T05:51:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtables.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph/libceph.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/failover.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/pktgen.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_diag.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv4.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv6.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver/dns_resolver.ko 2024-03-08T05:51:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/dsa_core.ko 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_brcm.ko 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_dsa.ko 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_none.ko 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_trailer.ko 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154.ko 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ah4.ko 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/esp4.ko 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/fou.ko 2024-03-08T05:51:59.302 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/gre.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/inet_diag.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_gre.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_tunnel.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_vti.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipcomp.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipip.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-08T05:51:59.303 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_bic.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_dctcp.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_diag.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_highspeed.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_htcp.ko 2024-03-08T05:51:59.304 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_hybla.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_illinois.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_lp.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_scalable.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_vegas.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_veno.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_westwood.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_yeah.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tunnel4.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_diag.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_tunnel.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ah6.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/esp6.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/fou6.ko 2024-03-08T05:51:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_tunnel.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_vti.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ipcomp6.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-08T05:51:59.306 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/sit.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/tunnel6.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key 2024-03-08T05:51:59.307 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key/af_key.ko 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_core.ko 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_eth.ko 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip.ko 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip6.ko 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_netlink.ko 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ppp.ko 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc/llc.ko 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154/mac802154.ko 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls/mpls_gso.ko 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter 2024-03-08T05:51:59.308 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-08T05:51:59.309 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-08T05:51:59.310 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conncount.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-08T05:51:59.311 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_log_syslog.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_irc.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_sip.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_tables.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_log.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-08T05:51:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_chain_nat.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_compat.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_ct.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fib.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_hash.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_limit.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_log.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_masq.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_nat.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_numgen.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_queue.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_quota.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_redir.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject_inet.ko 2024-03-08T05:51:59.313 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_socket.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_tunnel.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_AUDIT.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CT.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_DSCP.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HL.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HMARK.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LED.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LOG.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NETMAP.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFLOG.ko 2024-03-08T05:51:59.314 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_RATEEST.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_SECMARK.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TEE.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TPROXY.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TRACE.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_addrtype.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_bpf.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cgroup.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cluster.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_comment.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connbytes.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlabel.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlimit.ko 2024-03-08T05:51:59.315 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connmark.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_conntrack.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cpu.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dccp.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_devgroup.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dscp.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ecn.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_esp.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hashlimit.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_helper.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hl.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipcomp.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_iprange.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipvs.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_l2tp.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_length.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_limit.ko 2024-03-08T05:51:59.316 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mac.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mark.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_multiport.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nat.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nfacct.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_osf.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_owner.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_physdev.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_pkttype.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_policy.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_quota.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_rateest.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_realm.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_recent.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_sctp.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_set.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_socket.ko 2024-03-08T05:51:59.317 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_state.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_statistic.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_string.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_tcpmss.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_time.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_u32.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink/netlink_diag.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh/nsh.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/openvswitch.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-geneve.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-gre.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-vxlan.ko 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet 2024-03-08T05:51:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet/af_packet_diag.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_rdma.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_tcp.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_csum.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_gact.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_mirred.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_nat.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_pedit.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_police.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_simple.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_skbedit.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_vlan.ko 2024-03-08T05:51:59.319 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_basic.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_bpf.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_flow.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_fw.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_route.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_u32.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_cmp.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_ipset.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_meta.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_nbyte.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_text.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_u32.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_choke.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_codel.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_drr.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_fq.ko 2024-03-08T05:51:59.320 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_gred.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hfsc.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hhf.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_htb.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_ingress.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio_lib.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_multiq.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_netem.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_pie.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_plug.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_prio.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_qfq.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_red.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfb.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfq.ko 2024-03-08T05:51:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_tbf.ko 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_teql.ko 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp.ko 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp_diag.ko 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/sunrpc.ko 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/diag.ko 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/tipc.ko 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix 2024-03-08T05:51:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix/unix_diag.ko 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock.ko 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys/trusted.ko 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt 2024-03-08T05:51:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib/irqbypass.ko 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias.bin 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.alias.bin 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.bin 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.modinfo 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep.bin 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.devname 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.order 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.softdep 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols.bin 2024-03-08T05:51:59.324 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-08T05:51:59.327 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:51:59.328 DEBUG:teuthology.orchestra.run.smithi053:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-08T05:51:59.351 INFO:teuthology.orchestra.run.smithi053.stdout:postinstall scriptlet (using /bin/sh): 2024-03-08T05:51:59.351 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-08T05:51:59.351 INFO:teuthology.orchestra.run.smithi053.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gdd5c1cd17167 /lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-08T05:51:59.351 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2024-03-08T05:51:59.351 INFO:teuthology.orchestra.run.smithi053.stdout:for file in vmlinuz System.map config; do 2024-03-08T05:51:59.351 INFO:teuthology.orchestra.run.smithi053.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167"; then 2024-03-08T05:51:59.351 INFO:teuthology.orchestra.run.smithi053.stdout: cp "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167" 2024-03-08T05:51:59.351 INFO:teuthology.orchestra.run.smithi053.stdout: fi 2024-03-08T05:51:59.351 INFO:teuthology.orchestra.run.smithi053.stdout:done 2024-03-08T05:51:59.352 INFO:teuthology.orchestra.run.smithi053.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-08T05:51:59.352 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-08T05:51:59.352 INFO:teuthology.orchestra.run.smithi053.stdout:new-kernel-pkg --remove 6.8.0-rc7-gdd5c1cd17167 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gdd5c1cd17167.img 2024-03-08T05:51:59.352 INFO:teuthology.orchestra.run.smithi053.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-08T05:51:59.352 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-install remove 6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:51:59.352 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2024-03-08T05:51:59.352 INFO:teuthology.orchestra.run.smithi053.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-08T05:51:59.352 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-08T05:51:59.352 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:51:59.352 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2024-03-08T05:51:59.353 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -qi grub2-tools 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Name : grub2-tools 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Epoch : 1 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Version : 2.06 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Release : 68.el9 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: x86_64 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Group : Unspecified 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Size : 8264254 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:License : GPLv3+ 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-08T05:51:59.433 INFO:teuthology.orchestra.run.smithi053.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:Packager : builder@centos.org 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:Vendor : CentOS 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:Summary : Support tools for GRUB. 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:Description : 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:hardware devices. 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T05:51:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:This subpackage provides tools for support of all platforms. 2024-03-08T05:51:59.435 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-08T05:51:59.435 INFO:teuthology.task.kernel:Updating grub on smithi053 to boot 6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:51:59.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-08T05:52:00.051 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2024-03-08T05:52:01.509 INFO:teuthology.orchestra.run.smithi053.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-08T05:52:01.558 INFO:teuthology.orchestra.run.smithi053.stderr:done 2024-03-08T05:52:01.560 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-03-08T05:52:01.576 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.3r4PeYfKoE 2024-03-08T05:52:01.577 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3r4PeYfKoE 2024-03-08T05:52:01.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.3r4PeYfKoE 2024-03-08T05:52:01.796 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.3r4PeYfKoE is 6KB 2024-03-08T05:52:01.845 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.3r4PeYfKoE 2024-03-08T05:52:01.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo /bin/ls /boot/loader/entries || true 2024-03-08T05:52:01.929 INFO:teuthology.orchestra.run.smithi053.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-08T05:52:01.929 INFO:teuthology.orchestra.run.smithi053.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-08T05:52:01.930 INFO:teuthology.orchestra.run.smithi053.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-08T05:52:01.930 INFO:teuthology.orchestra.run.smithi053.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167.conf 2024-03-08T05:52:01.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:52:02.062 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2024-03-08T05:52:02.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-03-08T05:52:02.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:52:32.094 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-08T05:52:32.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-03-08T05:52:32.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:52:50.467 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-03-08T05:52:59.474 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-03-08T05:52:59.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:53:02.383 DEBUG:teuthology.orchestra.remote:timed out 2024-03-08T05:53:14.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-03-08T05:53:14.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:53:14.730 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-03-08T05:53:15.343 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-03-08T05:53:15.343 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc7-gdd5c1cd17167"... 2024-03-08T05:53:15.343 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-03-08T05:53:15.396 INFO:teuthology.orchestra.run.smithi040.stdout:6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:53:15.397 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gdd5c1cd17167 vs 6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:53:15.397 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-08T05:53:15.397 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-08T05:53:16.398 DEBUG:teuthology.parallel:result is None 2024-03-08T05:53:17.858 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-03-08T05:53:29.859 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-03-08T05:53:29.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:53:32.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-03-08T05:53:47.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-03-08T05:53:47.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T05:53:48.712 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-03-08T05:53:49.291 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-03-08T05:53:49.291 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.8.0-rc7-gdd5c1cd17167"... 2024-03-08T05:53:49.292 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-03-08T05:53:49.332 INFO:teuthology.orchestra.run.smithi053.stdout:6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:53:49.332 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gdd5c1cd17167 vs 6.8.0-rc7-gdd5c1cd17167 2024-03-08T05:53:49.332 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-08T05:53:49.332 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-08T05:53:50.333 DEBUG:teuthology.parallel:result is None 2024-03-08T05:53:50.334 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-08T05:53:50.345 INFO:teuthology.task.internal:Creating test directory... 2024-03-08T05:53:50.345 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T05:53:50.353 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T05:53:50.403 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-08T05:53:50.410 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-08T05:53:50.431 INFO:teuthology.task.internal:Creating archive directory... 2024-03-08T05:53:50.431 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T05:53:50.438 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T05:53:50.505 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-08T05:53:50.512 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-08T05:53:50.512 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T05:53:50.519 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T05:53:50.691 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T05:53:50.697 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T05:53:50.768 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T05:53:50.779 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T05:53:50.790 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-08T05:53:50.798 INFO:teuthology.task.internal:Configuring sudo... 2024-03-08T05:53:50.798 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T05:53:50.805 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T05:53:50.950 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-08T05:53:50.982 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-08T05:53:50.982 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T05:53:50.991 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T05:53:51.067 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T05:53:51.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T05:53:51.269 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T05:53:51.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T05:53:51.462 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T05:53:51.462 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T05:53:51.560 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T05:53:51.670 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T05:53:51.815 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T05:53:51.928 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T05:53:52.066 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T05:53:52.066 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T05:53:52.211 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-03-08T05:53:52.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-03-08T05:53:52.355 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T05:53:52.424 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T05:53:52.874 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-08T05:53:52.881 INFO:teuthology.task.internal:Starting timer... 2024-03-08T05:53:52.882 INFO:teuthology.run_tasks:Running task pcp... 2024-03-08T05:53:52.926 INFO:teuthology.run_tasks:Running task selinux... 2024-03-08T05:53:52.959 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2024-03-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2024-03-08T05:53:53.102 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-03-08T05:53:53.229 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-03-08T05:53:53.233 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-08T05:53:53.233 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2024-03-08T05:53:53.283 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2024-03-08T05:53:53.284 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-03-08T05:53:53.341 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-03-08T05:53:53.342 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi053.front.sepia.ceph.com': 'permissive'} 2024-03-08T05:53:53.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T05:53:53.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T05:53:53.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T05:53:53.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T05:53:53.528 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-08T05:53:53.528 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2024-03-08T05:53:53.612 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-03-08T05:53:53.698 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-08T05:53:53.710 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-08T05:53:53.728 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-08T05:53:53.729 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi053.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-08T06:00:02.041 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com')] 2024-03-08T06:00:02.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-03-08T06:00:02.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T06:00:02.195 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-03-08T06:00:02.354 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-03-08T06:00:02.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-03-08T06:00:02.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T06:00:02.503 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-03-08T06:00:02.657 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-03-08T06:00:02.657 INFO:teuthology.run_tasks:Running task clock... 2024-03-08T06:00:02.671 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-08T06:00:02.671 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T06:00:02.671 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T06:00:02.679 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T06:00:02.679 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T06:00:02.778 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T06:00:02.786 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T06:00:02.835 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T06:00:02.835 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T06:00:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2024-03-08T06:00:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-03-08T06:00:02.971 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2024-03-08T06:00:02.976 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-03-08T06:00:03.028 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T06:00:03.032 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T06:00:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T06:00:03.090 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T06:00:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-03-08T06:00:03.225 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T06:00:03.225 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-03-08T06:00:03.225 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:00:03.225 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:00:03.225 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:00:03.225 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:00:03.227 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-03-08T06:00:03.236 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T06:00:03.237 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-03-08T06:00:03.237 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:00:03.237 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:00:03.237 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:00:03.237 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:00:03.238 INFO:teuthology.run_tasks:Running task install... 2024-03-08T06:00:03.246 DEBUG:teuthology.task.install:project ceph 2024-03-08T06:00:03.247 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}} 2024-03-08T06:00:03.247 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'} 2024-03-08T06:00:03.247 INFO:teuthology.task.install:Using flavor: default 2024-03-08T06:00:03.252 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-08T06:00:03.252 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-08T06:00:03.253 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'tag': None, 'wait_for_package': False} 2024-03-08T06:00:03.254 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T06:00:03.256 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'tag': None, 'wait_for_package': False} 2024-03-08T06:00:03.256 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T06:00:03.402 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/ 2024-03-08T06:00:03.403 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1670.g1fb5d106 2024-03-08T06:00:03.470 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/ 2024-03-08T06:00:03.470 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1670.g1fb5d106 2024-03-08T06:00:03.510 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T06:00:03.510 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:00:03.510 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T06:00:03.623 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T06:00:03.623 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1fb5d10608dccee73edd6ef8a2c7b804a097c324/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T06:00:03.640 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T06:00:03.640 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:00:03.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T06:00:03.742 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T06:00:03.743 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1fb5d10608dccee73edd6ef8a2c7b804a097c324/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T06:00:03.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T06:00:03.887 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T06:00:03.906 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T06:00:03.999 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-03-08T06:00:04.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-03-08T06:00:04.031 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T06:00:04.126 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2024-03-08T06:00:04.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2024-03-08T06:00:04.369 INFO:teuthology.orchestra.run.smithi053.stdout:74 files removed 2024-03-08T06:00:04.409 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T06:00:04.487 INFO:teuthology.orchestra.run.smithi040.stdout:74 files removed 2024-03-08T06:00:04.527 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T06:00:05.741 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 100 kB/s | 82 kB 00:00 2024-03-08T06:00:05.747 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 118 kB/s | 82 kB 00:00 2024-03-08T06:00:06.606 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 13 kB/s | 10 kB 00:00 2024-03-08T06:00:06.608 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 13 kB/s | 10 kB 00:00 2024-03-08T06:00:07.476 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-03-08T06:00:07.482 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-03-08T06:00:08.093 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-08T06:00:08.396 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - AppStream 7.9 MB/s | 6.7 MB 00:00 2024-03-08T06:00:09.802 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2024-03-08T06:00:10.172 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2024-03-08T06:00:11.443 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-03-08T06:00:13.518 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-03-08T06:00:14.240 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.0 MB 00:04 2024-03-08T06:00:17.708 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 19 MB 00:02 2024-03-08T06:00:18.069 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-08T06:00:19.887 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 15 kB 00:00 2024-03-08T06:00:21.978 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-08T06:00:27.955 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-08T06:00:30.082 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 743 kB/s | 6.1 MB 00:08 2024-03-08T06:00:30.785 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T06:00:30.787 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T06:00:30.812 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T06:00:30.814 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T06:00:30.815 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T06:00:30.817 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T06:00:30.820 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T06:00:30.824 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T06:00:30.825 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T06:00:30.920 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-03-08T06:00:30.937 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 6.6 k 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.3 M 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 887 k 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 145 k 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 1.5 M 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 138 k 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 1.8 M 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 7.4 M 2024-03-08T06:00:30.938 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 49 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 9.8 M 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 47 M 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 264 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 748 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 220 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 27 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 18 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 31 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 745 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 127 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T06:00:30.939 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 162 k 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 320 k 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 298 k 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 99 k 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 85 k 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 170 k 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-03-08T06:00:30.940 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.4 M 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T06:00:30.941 INFO:teuthology.orchestra.run.smithi053.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 20 M 2024-03-08T06:00:30.942 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 31 k 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 2.1 M 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 248 k 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 4.7 M 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 17 M 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 15 k 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 25 k 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T06:00:30.943 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 163 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 501 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.1 M 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T06:00:30.944 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 45 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 131 k 2024-03-08T06:00:30.945 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T06:00:30.946 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T06:00:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T06:00:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T06:00:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T06:00:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T06:00:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T06:00:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T06:00:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T06:00:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T06:00:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T06:00:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T06:00:30.947 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T06:00:30.948 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T06:00:30.949 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-03-08T06:00:30.950 INFO:teuthology.orchestra.run.smithi053.stdout:Install 150 Packages 2024-03-08T06:00:30.951 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 24 Packages 2024-03-08T06:00:30.951 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:00:30.951 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 223 M 2024-03-08T06:00:30.951 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-03-08T06:00:31.633 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 15 kB 00:00 2024-03-08T06:00:32.373 INFO:teuthology.orchestra.run.smithi053.stdout:(1/174): ceph-19.0.0-1670.g1fb5d106.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-03-08T06:00:32.582 INFO:teuthology.orchestra.run.smithi053.stdout:(2/174): ceph-fuse-19.0.0-1670.g1fb5d106.el9.x8 4.2 MB/s | 887 kB 00:00 2024-03-08T06:00:32.640 INFO:teuthology.orchestra.run.smithi053.stdout:(3/174): ceph-immutable-object-cache-19.0.0-167 2.5 MB/s | 145 kB 00:00 2024-03-08T06:00:32.818 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-03-08T06:00:32.916 INFO:teuthology.orchestra.run.smithi053.stdout:(4/174): ceph-mds-19.0.0-1670.g1fb5d106.el9.x86 7.8 MB/s | 2.1 MB 00:00 2024-03-08T06:00:33.032 INFO:teuthology.orchestra.run.smithi053.stdout:(5/174): ceph-base-19.0.0-1670.g1fb5d106.el9.x8 5.6 MB/s | 5.3 MB 00:00 2024-03-08T06:00:33.116 INFO:teuthology.orchestra.run.smithi053.stdout:(6/174): ceph-mgr-19.0.0-1670.g1fb5d106.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-03-08T06:00:33.525 INFO:teuthology.orchestra.run.smithi053.stdout:(7/174): ceph-mon-19.0.0-1670.g1fb5d106.el9.x86 9.5 MB/s | 4.7 MB 00:00 2024-03-08T06:00:34.717 INFO:teuthology.orchestra.run.smithi053.stdout:(8/174): ceph-radosgw-19.0.0-1670.g1fb5d106.el9 8.2 MB/s | 9.8 MB 00:01 2024-03-08T06:00:34.784 INFO:teuthology.orchestra.run.smithi053.stdout:(9/174): ceph-selinux-19.0.0-1670.g1fb5d106.el9 384 kB/s | 25 kB 00:00 2024-03-08T06:00:35.043 INFO:teuthology.orchestra.run.smithi053.stdout:(10/174): ceph-osd-19.0.0-1670.g1fb5d106.el9.x8 8.7 MB/s | 17 MB 00:01 2024-03-08T06:00:35.109 INFO:teuthology.orchestra.run.smithi053.stdout:(11/174): cephfs-mirror-19.0.0-1670.g1fb5d106.e 3.3 MB/s | 220 kB 00:00 2024-03-08T06:00:35.160 INFO:teuthology.orchestra.run.smithi053.stdout:(12/174): cephfs-shell-19.0.0-1670.g1fb5d106.el 551 kB/s | 27 kB 00:00 2024-03-08T06:00:35.462 INFO:teuthology.orchestra.run.smithi053.stdout:(13/174): ceph-common-19.0.0-1670.g1fb5d106.el9 5.9 MB/s | 20 MB 00:03 2024-03-08T06:00:35.485 INFO:teuthology.orchestra.run.smithi053.stdout:(14/174): libcephfs-devel-19.0.0-1670.g1fb5d106 96 kB/s | 31 kB 00:00 2024-03-08T06:00:35.536 INFO:teuthology.orchestra.run.smithi053.stdout:(15/174): libcephsqlite-19.0.0-1670.g1fb5d106.e 3.2 MB/s | 163 kB 00:00 2024-03-08T06:00:35.611 INFO:teuthology.orchestra.run.smithi053.stdout:(16/174): librados-devel-19.0.0-1670.g1fb5d106. 1.7 MB/s | 127 kB 00:00 2024-03-08T06:00:35.661 INFO:teuthology.orchestra.run.smithi053.stdout:(17/174): libcephfs2-19.0.0-1670.g1fb5d106.el9. 3.7 MB/s | 745 kB 00:00 2024-03-08T06:00:35.753 INFO:teuthology.orchestra.run.smithi053.stdout:(18/174): libradosstriper1-19.0.0-1670.g1fb5d10 3.5 MB/s | 501 kB 00:00 2024-03-08T06:00:35.891 INFO:teuthology.orchestra.run.smithi053.stdout:(19/174): python3-ceph-argparse-19.0.0-1670.g1f 329 kB/s | 45 kB 00:00 2024-03-08T06:00:36.004 INFO:teuthology.orchestra.run.smithi053.stdout:(20/174): python3-ceph-common-19.0.0-1670.g1fb5 1.2 MB/s | 131 kB 00:00 2024-03-08T06:00:36.121 INFO:teuthology.orchestra.run.smithi053.stdout:(21/174): python3-cephfs-19.0.0-1670.g1fb5d106. 1.4 MB/s | 162 kB 00:00 2024-03-08T06:00:36.488 INFO:teuthology.orchestra.run.smithi053.stdout:(22/174): librgw2-19.0.0-1670.g1fb5d106.el9.x86 6.2 MB/s | 5.1 MB 00:00 2024-03-08T06:00:36.580 INFO:teuthology.orchestra.run.smithi053.stdout:(23/174): python3-rbd-19.0.0-1670.g1fb5d106.el9 3.2 MB/s | 298 kB 00:00 2024-03-08T06:00:36.622 INFO:teuthology.orchestra.run.smithi053.stdout:(24/174): python3-rados-19.0.0-1670.g1fb5d106.e 641 kB/s | 320 kB 00:00 2024-03-08T06:00:36.655 INFO:teuthology.orchestra.run.smithi053.stdout:(25/174): python3-rgw-19.0.0-1670.g1fb5d106.el9 1.3 MB/s | 99 kB 00:00 2024-03-08T06:00:36.714 INFO:teuthology.orchestra.run.smithi053.stdout:(26/174): rbd-fuse-19.0.0-1670.g1fb5d106.el9.x8 936 kB/s | 85 kB 00:00 2024-03-08T06:00:36.806 INFO:teuthology.orchestra.run.smithi053.stdout:(27/174): rbd-nbd-19.0.0-1670.g1fb5d106.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-08T06:00:36.898 INFO:teuthology.orchestra.run.smithi053.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1670.g 346 kB/s | 31 kB 00:00 2024-03-08T06:00:37.215 INFO:teuthology.orchestra.run.smithi053.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1670.g1fb5d10 437 kB/s | 138 kB 00:00 2024-03-08T06:00:37.299 INFO:teuthology.orchestra.run.smithi053.stdout:(30/174): rbd-mirror-19.0.0-1670.g1fb5d106.el9. 4.8 MB/s | 3.1 MB 00:00 2024-03-08T06:00:37.682 INFO:teuthology.orchestra.run.smithi053.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1670.g1fb5d 3.8 MB/s | 1.8 MB 00:00 2024-03-08T06:00:37.782 INFO:teuthology.orchestra.run.smithi053.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1670.g1f 2.5 MB/s | 248 kB 00:00 2024-03-08T06:00:37.883 INFO:teuthology.orchestra.run.smithi053.stdout:(33/174): ceph-mgr-rook-19.0.0-1670.g1fb5d106.e 497 kB/s | 49 kB 00:00 2024-03-08T06:00:37.975 INFO:teuthology.orchestra.run.smithi053.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1670.g1 169 kB/s | 15 kB 00:00 2024-03-08T06:00:38.084 INFO:teuthology.orchestra.run.smithi053.stdout:(35/174): ceph-volume-19.0.0-1670.g1fb5d106.el9 2.4 MB/s | 264 kB 00:00 2024-03-08T06:00:38.392 INFO:teuthology.orchestra.run.smithi053.stdout:(36/174): cephadm-19.0.0-1670.g1fb5d106.el9.noa 2.4 MB/s | 748 kB 00:00 2024-03-08T06:00:38.497 INFO:teuthology.orchestra.run.smithi053.stdout:(37/174): cephfs-top-19.0.0-1670.g1fb5d106.el9. 176 kB/s | 18 kB 00:00 2024-03-08T06:00:38.685 INFO:teuthology.orchestra.run.smithi053.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-03-08T06:00:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-08T06:00:38.835 INFO:teuthology.orchestra.run.smithi053.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 6.2 MB/s | 920 kB 00:00 2024-03-08T06:00:38.877 INFO:teuthology.orchestra.run.smithi053.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 662 kB 00:00 2024-03-08T06:00:38.902 INFO:teuthology.orchestra.run.smithi053.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.7 MB/s | 106 kB 00:00 2024-03-08T06:00:38.969 INFO:teuthology.orchestra.run.smithi053.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.4 MB/s | 307 kB 00:00 2024-03-08T06:00:38.986 INFO:teuthology.orchestra.run.smithi053.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 395 kB/s | 32 kB 00:00 2024-03-08T06:00:39.103 INFO:teuthology.orchestra.run.smithi053.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2024-03-08T06:00:39.120 INFO:teuthology.orchestra.run.smithi053.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 160 kB/s | 17 kB 00:00 2024-03-08T06:00:39.154 INFO:teuthology.orchestra.run.smithi053.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 313 kB/s | 10 kB 00:00 2024-03-08T06:00:39.187 INFO:teuthology.orchestra.run.smithi053.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 159 kB 00:00 2024-03-08T06:00:39.204 INFO:teuthology.orchestra.run.smithi053.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 933 kB/s | 44 kB 00:00 2024-03-08T06:00:39.246 INFO:teuthology.orchestra.run.smithi053.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-08T06:00:39.288 INFO:teuthology.orchestra.run.smithi053.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.4 MB/s | 578 kB 00:00 2024-03-08T06:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 775 kB/s | 46 kB 00:00 2024-03-08T06:00:39.378 INFO:teuthology.orchestra.run.smithi053.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 282 kB 00:00 2024-03-08T06:00:39.522 INFO:teuthology.orchestra.run.smithi053.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 910 kB/s | 187 kB 00:00 2024-03-08T06:00:39.556 INFO:teuthology.orchestra.run.smithi053.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.2 MB/s | 217 kB 00:00 2024-03-08T06:00:39.581 INFO:teuthology.orchestra.run.smithi053.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 34 kB 00:00 2024-03-08T06:00:39.765 INFO:teuthology.orchestra.run.smithi053.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-03-08T06:00:39.849 INFO:teuthology.orchestra.run.smithi053.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 6.3 MB/s | 1.5 MB 00:00 2024-03-08T06:00:39.899 INFO:teuthology.orchestra.run.smithi053.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 6.0 MB/s | 811 kB 00:00 2024-03-08T06:00:39.924 INFO:teuthology.orchestra.run.smithi053.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 1.9 MB/s | 134 kB 00:00 2024-03-08T06:00:40.092 INFO:teuthology.orchestra.run.smithi053.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2024-03-08T06:00:40.133 INFO:teuthology.orchestra.run.smithi053.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.7 MB/s | 1.0 MB 00:00 2024-03-08T06:00:40.183 INFO:teuthology.orchestra.run.smithi053.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.6 MB/s | 78 kB 00:00 2024-03-08T06:00:40.334 INFO:teuthology.orchestra.run.smithi053.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-03-08T06:00:40.359 INFO:teuthology.orchestra.run.smithi053.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 167 kB/s | 24 kB 00:00 2024-03-08T06:00:40.401 INFO:teuthology.orchestra.run.smithi053.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 448 kB/s | 18 kB 00:00 2024-03-08T06:00:40.426 INFO:teuthology.orchestra.run.smithi053.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-03-08T06:00:40.451 INFO:teuthology.orchestra.run.smithi053.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 653 kB/s | 32 kB 00:00 2024-03-08T06:00:40.593 INFO:teuthology.orchestra.run.smithi053.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-03-08T06:00:40.618 INFO:teuthology.orchestra.run.smithi053.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-08T06:00:40.694 INFO:teuthology.orchestra.run.smithi053.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 932 kB/s | 69 kB 00:00 2024-03-08T06:00:40.719 INFO:teuthology.orchestra.run.smithi053.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 174 kB 00:00 2024-03-08T06:00:40.744 INFO:teuthology.orchestra.run.smithi053.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 854 kB/s | 41 kB 00:00 2024-03-08T06:00:40.778 INFO:teuthology.orchestra.run.smithi053.stdout:(73/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.9 MB/s | 132 kB 00:00 2024-03-08T06:00:40.812 INFO:teuthology.orchestra.run.smithi053.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 238 kB 00:00 2024-03-08T06:00:40.845 INFO:teuthology.orchestra.run.smithi053.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 211 kB 00:00 2024-03-08T06:00:40.862 INFO:teuthology.orchestra.run.smithi053.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 996 kB/s | 47 kB 00:00 2024-03-08T06:00:40.880 INFO:teuthology.orchestra.run.smithi053.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-08T06:00:41.071 INFO:teuthology.orchestra.run.smithi053.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 239 kB/s | 39 kB 00:00 2024-03-08T06:00:41.247 INFO:teuthology.orchestra.run.smithi053.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-03-08T06:00:41.264 INFO:teuthology.orchestra.run.smithi053.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 218 kB/s | 41 kB 00:00 2024-03-08T06:00:41.297 INFO:teuthology.orchestra.run.smithi053.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 300 kB 00:00 2024-03-08T06:00:41.314 INFO:teuthology.orchestra.run.smithi053.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.2 MB/s | 56 kB 00:00 2024-03-08T06:00:41.339 INFO:teuthology.orchestra.run.smithi053.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-08T06:00:41.357 INFO:teuthology.orchestra.run.smithi053.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-03-08T06:00:41.382 INFO:teuthology.orchestra.run.smithi053.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.9 MB/s | 79 kB 00:00 2024-03-08T06:00:41.415 INFO:teuthology.orchestra.run.smithi053.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 241 kB 00:00 2024-03-08T06:00:41.474 INFO:teuthology.orchestra.run.smithi053.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-08T06:00:41.499 INFO:teuthology.orchestra.run.smithi053.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-08T06:00:41.525 INFO:teuthology.orchestra.run.smithi053.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-08T06:00:41.575 INFO:teuthology.orchestra.run.smithi053.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 187 kB 00:00 2024-03-08T06:00:41.638 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T06:00:41.640 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T06:00:41.665 INFO:teuthology.orchestra.run.smithi040.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T06:00:41.667 INFO:teuthology.orchestra.run.smithi040.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T06:00:41.668 INFO:teuthology.orchestra.run.smithi040.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T06:00:41.670 INFO:teuthology.orchestra.run.smithi040.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T06:00:41.672 INFO:teuthology.orchestra.run.smithi040.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T06:00:41.676 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T06:00:41.678 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T06:00:41.777 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-03-08T06:00:41.799 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================================ 2024-03-08T06:00:41.799 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-03-08T06:00:41.799 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================================ 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 6.6 k 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.3 M 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 887 k 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 145 k 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 1.5 M 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 138 k 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 1.8 M 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 7.4 M 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 49 k 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 9.8 M 2024-03-08T06:00:41.800 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 47 M 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 264 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 748 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 220 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 27 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 18 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 31 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 745 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 127 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T06:00:41.801 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 162 k 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 320 k 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 298 k 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 99 k 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 85 k 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 170 k 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T06:00:41.802 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.4 M 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T06:00:41.803 INFO:teuthology.orchestra.run.smithi040.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 20 M 2024-03-08T06:00:41.804 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 31 k 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 2.1 M 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 248 k 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 4.7 M 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 17 M 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 15 k 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 25 k 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T06:00:41.805 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 163 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 501 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.1 M 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T06:00:41.806 INFO:teuthology.orchestra.run.smithi040.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T06:00:41.807 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 45 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 131 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T06:00:41.808 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T06:00:41.809 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T06:00:41.810 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T06:00:41.811 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-03-08T06:00:41.812 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================================ 2024-03-08T06:00:41.813 INFO:teuthology.orchestra.run.smithi040.stdout:Install 150 Packages 2024-03-08T06:00:41.813 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 24 Packages 2024-03-08T06:00:41.813 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:00:41.813 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 223 M 2024-03-08T06:00:41.813 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-03-08T06:00:42.184 INFO:teuthology.orchestra.run.smithi053.stdout:(91/174): ceph-test-19.0.0-1670.g1fb5d106.el9.x 6.4 MB/s | 47 MB 00:07 2024-03-08T06:00:42.326 INFO:teuthology.orchestra.run.smithi053.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 234 kB/s | 187 kB 00:00 2024-03-08T06:00:42.426 INFO:teuthology.orchestra.run.smithi053.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 945 kB/s | 803 kB 00:00 2024-03-08T06:00:42.470 INFO:teuthology.orchestra.run.smithi053.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 888 kB/s | 125 kB 00:00 2024-03-08T06:00:42.502 INFO:teuthology.orchestra.run.smithi053.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 625 kB/s | 195 kB 00:00 2024-03-08T06:00:42.560 INFO:teuthology.orchestra.run.smithi053.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 162 kB/s | 21 kB 00:00 2024-03-08T06:00:42.585 INFO:teuthology.orchestra.run.smithi053.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 223 kB/s | 25 kB 00:00 2024-03-08T06:00:42.686 INFO:teuthology.orchestra.run.smithi053.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-03-08T06:00:42.711 INFO:teuthology.orchestra.run.smithi053.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.0 MB/s | 248 kB 00:00 2024-03-08T06:00:42.745 INFO:teuthology.orchestra.run.smithi053.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.7 MB/s | 216 kB 00:00 2024-03-08T06:00:42.770 INFO:teuthology.orchestra.run.smithi053.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 604 kB/s | 35 kB 00:00 2024-03-08T06:00:42.795 INFO:teuthology.orchestra.run.smithi053.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 902 kB/s | 44 kB 00:00 2024-03-08T06:00:42.829 INFO:teuthology.orchestra.run.smithi053.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-08T06:00:42.916 INFO:teuthology.orchestra.run.smithi053.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 187 kB/s | 22 kB 00:00 2024-03-08T06:00:43.079 INFO:teuthology.orchestra.run.smithi053.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-03-08T06:00:43.162 INFO:teuthology.orchestra.run.smithi040.stdout:(1/174): ceph-19.0.0-1670.g1fb5d106.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-03-08T06:00:43.221 INFO:teuthology.orchestra.run.smithi053.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-03-08T06:00:43.288 INFO:teuthology.orchestra.run.smithi053.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 295 kB/s | 19 kB 00:00 2024-03-08T06:00:43.373 INFO:teuthology.orchestra.run.smithi053.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-03-08T06:00:43.497 INFO:teuthology.orchestra.run.smithi053.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 209 kB/s | 26 kB 00:00 2024-03-08T06:00:43.581 INFO:teuthology.orchestra.run.smithi053.stdout:(110/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.8 MB/s | 4.1 MB 00:01 2024-03-08T06:00:43.673 INFO:teuthology.orchestra.run.smithi053.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 278 kB/s | 49 kB 00:00 2024-03-08T06:00:43.795 INFO:teuthology.orchestra.run.smithi053.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-03-08T06:00:43.823 INFO:teuthology.orchestra.run.smithi053.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 279 kB/s | 67 kB 00:00 2024-03-08T06:00:43.849 INFO:teuthology.orchestra.run.smithi053.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 868 kB/s | 151 kB 00:00 2024-03-08T06:00:43.932 INFO:teuthology.orchestra.run.smithi053.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.0 MB/s | 838 kB 00:00 2024-03-08T06:00:44.016 INFO:teuthology.orchestra.run.smithi053.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 261 kB/s | 43 kB 00:00 2024-03-08T06:00:44.083 INFO:teuthology.orchestra.run.smithi053.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 215 kB/s | 32 kB 00:00 2024-03-08T06:00:44.095 INFO:teuthology.orchestra.run.smithi040.stdout:(2/174): ceph-base-19.0.0-1670.g1fb5d106.el9.x8 4.1 MB/s | 5.3 MB 00:01 2024-03-08T06:00:44.125 INFO:teuthology.orchestra.run.smithi053.stdout:(118/174): python3-asyncssh-2.13.2-4.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-03-08T06:00:44.184 INFO:teuthology.orchestra.run.smithi053.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 90 kB/s | 15 kB 00:00 2024-03-08T06:00:44.187 INFO:teuthology.orchestra.run.smithi040.stdout:(3/174): ceph-immutable-object-cache-19.0.0-167 1.6 MB/s | 145 kB 00:00 2024-03-08T06:00:44.245 INFO:teuthology.orchestra.run.smithi040.stdout:(4/174): ceph-fuse-19.0.0-1670.g1fb5d106.el9.x8 819 kB/s | 887 kB 00:01 2024-03-08T06:00:44.259 INFO:teuthology.orchestra.run.smithi053.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 981 kB/s | 172 kB 00:00 2024-03-08T06:00:44.293 INFO:teuthology.orchestra.run.smithi053.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-03-08T06:00:44.418 INFO:teuthology.orchestra.run.smithi053.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-08T06:00:44.443 INFO:teuthology.orchestra.run.smithi053.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 192 kB/s | 35 kB 00:00 2024-03-08T06:00:44.477 INFO:teuthology.orchestra.run.smithi053.stdout:(124/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.0 MB/s | 303 kB 00:00 2024-03-08T06:00:44.585 INFO:teuthology.orchestra.run.smithi053.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-03-08T06:00:44.596 INFO:teuthology.orchestra.run.smithi040.stdout:(5/174): ceph-mds-19.0.0-1670.g1fb5d106.el9.x86 5.2 MB/s | 2.1 MB 00:00 2024-03-08T06:00:44.694 INFO:teuthology.orchestra.run.smithi053.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 49 kB/s | 11 kB 00:00 2024-03-08T06:00:44.719 INFO:teuthology.orchestra.run.smithi053.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 133 kB/s | 18 kB 00:00 2024-03-08T06:00:44.861 INFO:teuthology.orchestra.run.smithi053.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-03-08T06:00:44.887 INFO:teuthology.orchestra.run.smithi053.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 121 kB/s | 23 kB 00:00 2024-03-08T06:00:44.912 INFO:teuthology.orchestra.run.smithi053.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 101 kB/s | 19 kB 00:00 2024-03-08T06:00:44.954 INFO:teuthology.orchestra.run.smithi053.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 215 kB/s | 20 kB 00:00 2024-03-08T06:00:45.071 INFO:teuthology.orchestra.run.smithi053.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 294 kB/s | 46 kB 00:00 2024-03-08T06:00:45.121 INFO:teuthology.orchestra.run.smithi053.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-03-08T06:00:45.188 INFO:teuthology.orchestra.run.smithi053.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 338 kB/s | 79 kB 00:00 2024-03-08T06:00:45.213 INFO:teuthology.orchestra.run.smithi053.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 407 kB/s | 58 kB 00:00 2024-03-08T06:00:45.246 INFO:teuthology.orchestra.run.smithi040.stdout:(6/174): ceph-mgr-19.0.0-1670.g1fb5d106.el9.x86 1.5 MB/s | 1.5 MB 00:00 2024-03-08T06:00:45.289 INFO:teuthology.orchestra.run.smithi053.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-03-08T06:00:45.314 INFO:teuthology.orchestra.run.smithi053.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-03-08T06:00:45.388 INFO:teuthology.orchestra.run.smithi040.stdout:(7/174): ceph-mon-19.0.0-1670.g1fb5d106.el9.x86 5.9 MB/s | 4.7 MB 00:00 2024-03-08T06:00:45.414 INFO:teuthology.orchestra.run.smithi053.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 309 kB/s | 31 kB 00:00 2024-03-08T06:00:45.439 INFO:teuthology.orchestra.run.smithi053.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 399 kB/s | 90 kB 00:00 2024-03-08T06:00:45.490 INFO:teuthology.orchestra.run.smithi053.stdout:(140/174): python3-pyperclip-1.8.2-2.el9.noarch 141 kB/s | 28 kB 00:00 2024-03-08T06:00:45.582 INFO:teuthology.orchestra.run.smithi053.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-03-08T06:00:45.690 INFO:teuthology.orchestra.run.smithi053.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 236 kB/s | 59 kB 00:00 2024-03-08T06:00:45.715 INFO:teuthology.orchestra.run.smithi053.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 383 kB/s | 51 kB 00:00 2024-03-08T06:00:45.807 INFO:teuthology.orchestra.run.smithi053.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 985 kB/s | 90 kB 00:00 2024-03-08T06:00:45.825 INFO:teuthology.orchestra.run.smithi053.stdout:(145/174): python3-tempora-5.0.0-2.el9.noarch.r 107 kB/s | 36 kB 00:00 2024-03-08T06:00:45.908 INFO:teuthology.orchestra.run.smithi053.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-08T06:00:45.934 INFO:teuthology.orchestra.run.smithi053.stdout:(147/174): python3-webob-1.8.7-6.el9.noarch.rpm 947 kB/s | 230 kB 00:00 2024-03-08T06:00:45.992 INFO:teuthology.orchestra.run.smithi053.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 120 kB/s | 20 kB 00:00 2024-03-08T06:00:46.034 INFO:teuthology.orchestra.run.smithi053.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-03-08T06:00:46.097 INFO:teuthology.orchestra.run.smithi040.stdout:(8/174): ceph-common-19.0.0-1670.g1fb5d106.el9. 6.0 MB/s | 20 MB 00:03 2024-03-08T06:00:46.197 INFO:teuthology.orchestra.run.smithi040.stdout:(9/174): ceph-selinux-19.0.0-1670.g1fb5d106.el9 255 kB/s | 25 kB 00:00 2024-03-08T06:00:46.310 INFO:teuthology.orchestra.run.smithi053.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-03-08T06:00:46.485 INFO:teuthology.orchestra.run.smithi053.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.8 MB/s | 321 kB 00:00 2024-03-08T06:00:46.877 INFO:teuthology.orchestra.run.smithi053.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 5.9 MB/s | 2.3 MB 00:00 2024-03-08T06:00:46.930 INFO:teuthology.orchestra.run.smithi040.stdout:(10/174): ceph-radosgw-19.0.0-1670.g1fb5d106.el 6.3 MB/s | 9.8 MB 00:01 2024-03-08T06:00:47.010 INFO:teuthology.orchestra.run.smithi053.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 14 MB/s | 1.8 MB 00:00 2024-03-08T06:00:47.031 INFO:teuthology.orchestra.run.smithi040.stdout:(11/174): cephfs-mirror-19.0.0-1670.g1fb5d106.e 2.2 MB/s | 220 kB 00:00 2024-03-08T06:00:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 738 kB/s | 42 kB 00:00 2024-03-08T06:00:47.122 INFO:teuthology.orchestra.run.smithi040.stdout:(12/174): cephfs-shell-19.0.0-1670.g1fb5d106.el 298 kB/s | 27 kB 00:00 2024-03-08T06:00:47.161 INFO:teuthology.orchestra.run.smithi053.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 2.7 MB/s | 248 kB 00:00 2024-03-08T06:00:47.214 INFO:teuthology.orchestra.run.smithi040.stdout:(13/174): libcephfs-devel-19.0.0-1670.g1fb5d106 344 kB/s | 31 kB 00:00 2024-03-08T06:00:47.228 INFO:teuthology.orchestra.run.smithi053.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 71 kB 00:00 2024-03-08T06:00:47.303 INFO:teuthology.orchestra.run.smithi053.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 492 kB/s | 36 kB 00:00 2024-03-08T06:00:47.378 INFO:teuthology.orchestra.run.smithi053.stdout:(158/174): librbd1-19.0.0-1670.g1fb5d106.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-03-08T06:00:47.398 INFO:teuthology.orchestra.run.smithi040.stdout:(14/174): libcephfs2-19.0.0-1670.g1fb5d106.el9. 4.0 MB/s | 745 kB 00:00 2024-03-08T06:00:47.437 INFO:teuthology.orchestra.run.smithi053.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.4 MB/s | 192 kB 00:00 2024-03-08T06:00:47.482 INFO:teuthology.orchestra.run.smithi040.stdout:(15/174): libcephsqlite-19.0.0-1670.g1fb5d106.e 1.9 MB/s | 163 kB 00:00 2024-03-08T06:00:47.548 INFO:teuthology.orchestra.run.smithi040.stdout:(16/174): librados-devel-19.0.0-1670.g1fb5d106. 1.9 MB/s | 127 kB 00:00 2024-03-08T06:00:47.571 INFO:teuthology.orchestra.run.smithi053.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 375 kB/s | 23 kB 00:00 2024-03-08T06:00:47.638 INFO:teuthology.orchestra.run.smithi053.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-08T06:00:47.657 INFO:teuthology.orchestra.run.smithi040.stdout:(17/174): libradosstriper1-19.0.0-1670.g1fb5d10 4.6 MB/s | 501 kB 00:00 2024-03-08T06:00:47.671 INFO:teuthology.orchestra.run.smithi053.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 877 kB/s | 256 kB 00:00 2024-03-08T06:00:47.738 INFO:teuthology.orchestra.run.smithi053.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-08T06:00:47.773 INFO:teuthology.orchestra.run.smithi053.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-03-08T06:00:47.823 INFO:teuthology.orchestra.run.smithi053.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 766 kB/s | 64 kB 00:00 2024-03-08T06:00:47.848 INFO:teuthology.orchestra.run.smithi053.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 386 kB/s | 29 kB 00:00 2024-03-08T06:00:47.948 INFO:teuthology.orchestra.run.smithi053.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2024-03-08T06:00:48.083 INFO:teuthology.orchestra.run.smithi053.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 9.3 MB/s | 2.2 MB 00:00 2024-03-08T06:00:48.308 INFO:teuthology.orchestra.run.smithi053.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.4 MB/s | 2.3 MB 00:00 2024-03-08T06:00:48.350 INFO:teuthology.orchestra.run.smithi053.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 1.7 MB/s | 465 kB 00:00 2024-03-08T06:00:48.392 INFO:teuthology.orchestra.run.smithi053.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 376 kB/s | 31 kB 00:00 2024-03-08T06:00:48.399 INFO:teuthology.orchestra.run.smithi040.stdout:(18/174): librgw2-19.0.0-1670.g1fb5d106.el9.x86 6.9 MB/s | 5.1 MB 00:00 2024-03-08T06:00:48.459 INFO:teuthology.orchestra.run.smithi053.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 74 kB 00:00 2024-03-08T06:00:48.509 INFO:teuthology.orchestra.run.smithi053.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2024-03-08T06:00:48.566 INFO:teuthology.orchestra.run.smithi040.stdout:(19/174): python3-ceph-argparse-19.0.0-1670.g1f 272 kB/s | 45 kB 00:00 2024-03-08T06:00:48.642 INFO:teuthology.orchestra.run.smithi040.stdout:(20/174): python3-ceph-common-19.0.0-1670.g1fb5 1.7 MB/s | 131 kB 00:00 2024-03-08T06:00:48.717 INFO:teuthology.orchestra.run.smithi040.stdout:(21/174): python3-cephfs-19.0.0-1670.g1fb5d106. 2.1 MB/s | 162 kB 00:00 2024-03-08T06:00:48.792 INFO:teuthology.orchestra.run.smithi040.stdout:(22/174): python3-rados-19.0.0-1670.g1fb5d106.e 4.3 MB/s | 320 kB 00:00 2024-03-08T06:00:48.876 INFO:teuthology.orchestra.run.smithi040.stdout:(23/174): python3-rbd-19.0.0-1670.g1fb5d106.el9 3.5 MB/s | 298 kB 00:00 2024-03-08T06:00:48.943 INFO:teuthology.orchestra.run.smithi040.stdout:(24/174): python3-rgw-19.0.0-1670.g1fb5d106.el9 1.5 MB/s | 99 kB 00:00 2024-03-08T06:00:49.010 INFO:teuthology.orchestra.run.smithi040.stdout:(25/174): rbd-fuse-19.0.0-1670.g1fb5d106.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-08T06:00:49.544 INFO:teuthology.orchestra.run.smithi040.stdout:(26/174): ceph-osd-19.0.0-1670.g1fb5d106.el9.x8 3.9 MB/s | 17 MB 00:04 2024-03-08T06:00:49.567 INFO:teuthology.orchestra.run.smithi053.stdout:(174/174): librados2-19.0.0-1670.g1fb5d106.el9. 968 kB/s | 3.4 MB 00:03 2024-03-08T06:00:49.578 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-03-08T06:00:49.578 INFO:teuthology.orchestra.run.smithi053.stdout:Total 12 MB/s | 223 MB 00:18 2024-03-08T06:00:49.635 INFO:teuthology.orchestra.run.smithi040.stdout:(27/174): rbd-mirror-19.0.0-1670.g1fb5d106.el9. 4.9 MB/s | 3.1 MB 00:00 2024-03-08T06:00:49.669 INFO:teuthology.orchestra.run.smithi040.stdout:(28/174): rbd-nbd-19.0.0-1670.g1fb5d106.el9.x86 1.3 MB/s | 170 kB 00:00 2024-03-08T06:00:49.686 INFO:teuthology.orchestra.run.smithi040.stdout:(29/174): ceph-grafana-dashboards-19.0.0-1670.g 638 kB/s | 31 kB 00:00 2024-03-08T06:00:49.744 INFO:teuthology.orchestra.run.smithi040.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1670.g1fb5d10 1.8 MB/s | 138 kB 00:00 2024-03-08T06:00:49.987 INFO:teuthology.orchestra.run.smithi040.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1670.g1fb5d 5.9 MB/s | 1.8 MB 00:00 2024-03-08T06:00:50.095 INFO:teuthology.orchestra.run.smithi040.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1670.g1f 2.3 MB/s | 248 kB 00:00 2024-03-08T06:00:50.179 INFO:teuthology.orchestra.run.smithi040.stdout:(33/174): ceph-mgr-rook-19.0.0-1670.g1fb5d106.e 600 kB/s | 49 kB 00:00 2024-03-08T06:00:50.271 INFO:teuthology.orchestra.run.smithi040.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1670.g1 169 kB/s | 15 kB 00:00 2024-03-08T06:00:50.346 INFO:teuthology.orchestra.run.smithi040.stdout:(35/174): ceph-volume-19.0.0-1670.g1fb5d106.el9 3.5 MB/s | 264 kB 00:00 2024-03-08T06:00:50.488 INFO:teuthology.orchestra.run.smithi040.stdout:(36/174): cephadm-19.0.0-1670.g1fb5d106.el9.noa 5.2 MB/s | 748 kB 00:00 2024-03-08T06:00:50.555 INFO:teuthology.orchestra.run.smithi040.stdout:(37/174): cephfs-top-19.0.0-1670.g1fb5d106.el9. 278 kB/s | 18 kB 00:00 2024-03-08T06:00:50.764 INFO:teuthology.orchestra.run.smithi040.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2024-03-08T06:00:50.822 INFO:teuthology.orchestra.run.smithi040.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 920 kB 00:00 2024-03-08T06:00:50.864 INFO:teuthology.orchestra.run.smithi040.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 106 kB 00:00 2024-03-08T06:00:50.906 INFO:teuthology.orchestra.run.smithi040.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 307 kB 00:00 2024-03-08T06:00:50.940 INFO:teuthology.orchestra.run.smithi040.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 976 kB/s | 32 kB 00:00 2024-03-08T06:00:51.049 INFO:teuthology.orchestra.run.smithi040.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-08T06:00:51.080 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-03-08T06:00:51.191 INFO:teuthology.orchestra.run.smithi040.stdout:(44/174): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-03-08T06:00:51.216 INFO:teuthology.orchestra.run.smithi040.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 103 kB/s | 17 kB 00:00 2024-03-08T06:00:51.241 INFO:teuthology.orchestra.run.smithi040.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 205 kB/s | 10 kB 00:00 2024-03-08T06:00:51.303 INFO:teuthology.orchestra.run.smithi040.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.8 MB/s | 159 kB 00:00 2024-03-08T06:00:51.333 INFO:teuthology.orchestra.run.smithi040.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 44 kB 00:00 2024-03-08T06:00:51.383 INFO:teuthology.orchestra.run.smithi040.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.2 MB/s | 661 kB 00:00 2024-03-08T06:00:51.426 INFO:teuthology.orchestra.run.smithi040.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.6 MB/s | 578 kB 00:00 2024-03-08T06:00:51.451 INFO:teuthology.orchestra.run.smithi040.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 757 kB/s | 46 kB 00:00 2024-03-08T06:00:51.479 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-03-08T06:00:51.479 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-03-08T06:00:51.485 INFO:teuthology.orchestra.run.smithi040.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-08T06:00:51.518 INFO:teuthology.orchestra.run.smithi040.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-08T06:00:51.552 INFO:teuthology.orchestra.run.smithi040.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-08T06:00:51.577 INFO:teuthology.orchestra.run.smithi040.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 612 kB/s | 34 kB 00:00 2024-03-08T06:00:51.711 INFO:teuthology.orchestra.run.smithi040.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-03-08T06:00:51.761 INFO:teuthology.orchestra.run.smithi040.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 9.1 MB/s | 1.5 MB 00:00 2024-03-08T06:00:51.812 INFO:teuthology.orchestra.run.smithi040.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 8.0 MB/s | 811 kB 00:00 2024-03-08T06:00:51.837 INFO:teuthology.orchestra.run.smithi040.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 1.9 MB/s | 134 kB 00:00 2024-03-08T06:00:51.987 INFO:teuthology.orchestra.run.smithi040.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2024-03-08T06:00:52.029 INFO:teuthology.orchestra.run.smithi040.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.7 MB/s | 1.0 MB 00:00 2024-03-08T06:00:52.082 INFO:teuthology.orchestra.run.smithi040.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 78 kB 00:00 2024-03-08T06:00:52.213 INFO:teuthology.orchestra.run.smithi040.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 5.8 MB 00:00 2024-03-08T06:00:52.230 INFO:teuthology.orchestra.run.smithi040.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 182 kB/s | 24 kB 00:00 2024-03-08T06:00:52.272 INFO:teuthology.orchestra.run.smithi040.stdout:(65/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.5 MB/s | 144 kB 00:00 2024-03-08T06:00:52.331 INFO:teuthology.orchestra.run.smithi040.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 182 kB/s | 18 kB 00:00 2024-03-08T06:00:52.356 INFO:teuthology.orchestra.run.smithi040.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 381 kB/s | 32 kB 00:00 2024-03-08T06:00:52.489 INFO:teuthology.orchestra.run.smithi040.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-08T06:00:52.514 INFO:teuthology.orchestra.run.smithi040.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-08T06:00:52.557 INFO:teuthology.orchestra.run.smithi040.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 1.7 MB/s | 69 kB 00:00 2024-03-08T06:00:52.582 INFO:teuthology.orchestra.run.smithi040.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-03-08T06:00:52.607 INFO:teuthology.orchestra.run.smithi040.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 857 kB/s | 41 kB 00:00 2024-03-08T06:00:52.641 INFO:teuthology.orchestra.run.smithi040.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.1 MB/s | 238 kB 00:00 2024-03-08T06:00:52.666 INFO:teuthology.orchestra.run.smithi040.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-08T06:00:52.699 INFO:teuthology.orchestra.run.smithi040.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-08T06:00:52.725 INFO:teuthology.orchestra.run.smithi040.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 847 kB/s | 47 kB 00:00 2024-03-08T06:00:52.750 INFO:teuthology.orchestra.run.smithi040.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 870 kB/s | 43 kB 00:00 2024-03-08T06:00:53.351 INFO:teuthology.orchestra.run.smithi040.stdout:(78/174): ceph-test-19.0.0-1670.g1fb5d106.el9.x 6.6 MB/s | 47 MB 00:07 2024-03-08T06:00:53.576 INFO:teuthology.orchestra.run.smithi040.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-03-08T06:00:53.601 INFO:teuthology.orchestra.run.smithi040.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 46 kB/s | 39 kB 00:00 2024-03-08T06:00:53.626 INFO:teuthology.orchestra.run.smithi040.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 157 kB/s | 41 kB 00:00 2024-03-08T06:00:53.677 INFO:teuthology.orchestra.run.smithi040.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 300 kB 00:00 2024-03-08T06:00:53.702 INFO:teuthology.orchestra.run.smithi040.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 56 kB 00:00 2024-03-08T06:00:53.727 INFO:teuthology.orchestra.run.smithi040.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 637 kB/s | 63 kB 00:00 2024-03-08T06:00:53.769 INFO:teuthology.orchestra.run.smithi040.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-03-08T06:00:53.794 INFO:teuthology.orchestra.run.smithi040.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2024-03-08T06:00:53.819 INFO:teuthology.orchestra.run.smithi040.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-03-08T06:00:53.870 INFO:teuthology.orchestra.run.smithi040.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-08T06:00:53.903 INFO:teuthology.orchestra.run.smithi040.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-08T06:00:53.937 INFO:teuthology.orchestra.run.smithi040.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 124 kB 00:00 2024-03-08T06:00:53.954 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-03-08T06:00:53.955 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-03-08T06:00:53.979 INFO:teuthology.orchestra.run.smithi040.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 187 kB 00:00 2024-03-08T06:00:54.112 INFO:teuthology.orchestra.run.smithi040.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-08T06:00:54.138 INFO:teuthology.orchestra.run.smithi040.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-08T06:00:54.171 INFO:teuthology.orchestra.run.smithi040.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-08T06:00:54.188 INFO:teuthology.orchestra.run.smithi040.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 435 kB/s | 21 kB 00:00 2024-03-08T06:00:54.214 INFO:teuthology.orchestra.run.smithi040.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 620 kB/s | 25 kB 00:00 2024-03-08T06:00:54.255 INFO:teuthology.orchestra.run.smithi040.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-03-08T06:00:54.297 INFO:teuthology.orchestra.run.smithi040.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-03-08T06:00:54.332 INFO:teuthology.orchestra.run.smithi040.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-03-08T06:00:54.381 INFO:teuthology.orchestra.run.smithi040.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.6 MB/s | 216 kB 00:00 2024-03-08T06:00:54.407 INFO:teuthology.orchestra.run.smithi040.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 473 kB/s | 35 kB 00:00 2024-03-08T06:00:54.432 INFO:teuthology.orchestra.run.smithi040.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 900 kB/s | 44 kB 00:00 2024-03-08T06:00:54.457 INFO:teuthology.orchestra.run.smithi040.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-03-08T06:00:54.591 INFO:teuthology.orchestra.run.smithi040.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 10 MB/s | 4.1 MB 00:00 2024-03-08T06:00:54.783 INFO:teuthology.orchestra.run.smithi040.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 64 kB/s | 22 kB 00:00 2024-03-08T06:00:55.242 INFO:teuthology.orchestra.run.smithi040.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-03-08T06:00:55.409 INFO:teuthology.orchestra.run.smithi040.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 117 kB/s | 19 kB 00:00 2024-03-08T06:00:56.575 INFO:teuthology.orchestra.run.smithi040.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 172 kB/s | 309 kB 00:01 2024-03-08T06:00:56.784 INFO:teuthology.orchestra.run.smithi040.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 125 kB/s | 26 kB 00:00 2024-03-08T06:00:57.109 INFO:teuthology.orchestra.run.smithi040.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 150 kB/s | 49 kB 00:00 2024-03-08T06:00:57.618 INFO:teuthology.orchestra.run.smithi040.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 133 kB/s | 67 kB 00:00 2024-03-08T06:00:58.535 INFO:teuthology.orchestra.run.smithi040.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 165 kB/s | 151 kB 00:00 2024-03-08T06:00:59.777 INFO:teuthology.orchestra.run.smithi040.stdout:(113/174): abseil-cpp-20211102.0-3.el9.x86_64.r 106 kB/s | 548 kB 00:05 2024-03-08T06:01:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-03-08T06:01:01.909 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T06:01:01.949 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T06:01:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T06:01:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T06:01:02.141 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T06:01:02.323 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T06:01:02.373 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T06:01:02.411 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T06:01:02.503 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T06:01:02.861 INFO:teuthology.orchestra.run.smithi040.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 194 kB/s | 838 kB 00:04 2024-03-08T06:01:02.964 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T06:01:03.034 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T06:01:03.177 INFO:teuthology.orchestra.run.smithi040.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 138 kB/s | 43 kB 00:00 2024-03-08T06:01:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T06:01:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T06:01:03.349 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T06:01:03.392 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T06:01:03.444 INFO:teuthology.orchestra.run.smithi040.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 121 kB/s | 32 kB 00:00 2024-03-08T06:01:03.502 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T06:01:03.594 INFO:teuthology.orchestra.run.smithi040.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 100 kB/s | 15 kB 00:00 2024-03-08T06:01:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T06:01:03.960 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T06:01:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T06:01:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T06:01:04.270 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T06:01:04.315 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T06:01:04.347 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T06:01:04.421 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T06:01:04.703 INFO:teuthology.orchestra.run.smithi040.stdout:(118/174): python3-asyncssh-2.13.2-4.el9.noarch 111 kB/s | 548 kB 00:04 2024-03-08T06:01:04.728 INFO:teuthology.orchestra.run.smithi040.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 152 kB/s | 172 kB 00:01 2024-03-08T06:01:04.738 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T06:01:04.825 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T06:01:04.881 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T06:01:05.226 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T06:01:05.490 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T06:01:05.580 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T06:01:05.801 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T06:01:05.884 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T06:01:06.291 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T06:01:06.492 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T06:01:06.554 INFO:teuthology.orchestra.run.smithi040.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 166 kB/s | 303 kB 00:01 2024-03-08T06:01:06.647 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T06:01:06.775 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T06:01:06.814 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T06:01:06.846 INFO:teuthology.orchestra.run.smithi040.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 121 kB/s | 35 kB 00:00 2024-03-08T06:01:06.904 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T06:01:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T06:01:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T06:01:07.158 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T06:01:07.262 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T06:01:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T06:01:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T06:01:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T06:01:07.483 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T06:01:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T06:01:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 39/198 2024-03-08T06:01:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 40/198 2024-03-08T06:01:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 41/198 2024-03-08T06:01:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T06:01:07.940 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T06:01:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T06:01:08.079 INFO:teuthology.orchestra.run.smithi040.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 164 kB/s | 203 kB 00:01 2024-03-08T06:01:08.250 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T06:01:08.321 INFO:teuthology.orchestra.run.smithi040.stdout:(123/174): python3-cherrypy-18.6.1-2.el9.noarch 99 kB/s | 358 kB 00:03 2024-03-08T06:01:08.341 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T06:01:09.085 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T06:01:09.179 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T06:01:09.318 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T06:01:09.379 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T06:01:09.406 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T06:01:09.455 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T06:01:09.534 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T06:01:09.614 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 53/198 2024-03-08T06:01:09.665 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T06:01:09.813 INFO:teuthology.orchestra.run.smithi040.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 97 kB/s | 144 kB 00:01 2024-03-08T06:01:10.005 INFO:teuthology.orchestra.run.smithi040.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-03-08T06:01:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T06:01:10.255 INFO:teuthology.orchestra.run.smithi040.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 71 kB/s | 18 kB 00:00 2024-03-08T06:01:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T06:01:10.384 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T06:01:10.431 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T06:01:10.663 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T06:01:10.664 INFO:teuthology.orchestra.run.smithi040.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 57 kB/s | 23 kB 00:00 2024-03-08T06:01:10.713 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T06:01:10.789 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T06:01:10.886 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T06:01:10.914 INFO:teuthology.orchestra.run.smithi040.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 78 kB/s | 19 kB 00:00 2024-03-08T06:01:10.938 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T06:01:10.987 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T06:01:11.057 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T06:01:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T06:01:11.164 INFO:teuthology.orchestra.run.smithi040.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 79 kB/s | 20 kB 00:00 2024-03-08T06:01:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T06:01:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T06:01:11.453 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T06:01:11.509 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T06:01:11.555 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T06:01:11.615 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T06:01:11.760 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T06:01:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T06:01:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T06:01:11.886 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T06:01:11.919 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T06:01:12.090 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T06:01:12.276 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T06:01:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T06:01:12.479 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T06:01:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T06:01:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T06:01:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T06:01:12.942 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T06:01:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T06:01:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T06:01:13.212 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T06:01:13.250 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T06:01:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T06:01:13.336 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T06:01:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T06:01:13.447 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T06:01:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T06:01:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T06:01:13.749 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T06:01:13.814 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T06:01:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T06:01:14.004 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T06:01:14.416 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T06:01:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T06:01:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T06:01:14.601 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T06:01:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T06:01:14.844 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T06:01:15.865 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T06:01:15.924 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T06:01:15.966 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T06:01:16.037 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T06:01:16.190 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T06:01:16.234 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T06:01:16.483 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T06:01:16.530 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T06:01:16.636 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T06:01:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T06:01:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T06:01:19.031 INFO:teuthology.orchestra.run.smithi040.stdout:(130/174): python3-grpcio-1.46.7-10.el9.x86_64. 191 kB/s | 2.0 MB 00:10 2024-03-08T06:01:19.280 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T06:01:19.347 INFO:teuthology.orchestra.run.smithi040.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 147 kB/s | 46 kB 00:00 2024-03-08T06:01:19.367 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T06:01:19.422 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T06:01:19.451 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T06:01:19.493 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T06:01:19.823 INFO:teuthology.orchestra.run.smithi040.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 166 kB/s | 79 kB 00:00 2024-03-08T06:01:19.912 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T06:01:19.940 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T06:01:20.053 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T06:01:20.089 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 125/198 2024-03-08T06:01:20.145 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T06:01:20.282 INFO:teuthology.orchestra.run.smithi040.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 126 kB/s | 58 kB 00:00 2024-03-08T06:01:20.687 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T06:01:20.718 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T06:01:20.817 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T06:01:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 129/198 2024-03-08T06:01:21.165 INFO:teuthology.orchestra.run.smithi040.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 105 kB/s | 1.0 MB 00:09 2024-03-08T06:01:21.365 INFO:teuthology.orchestra.run.smithi040.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 82 kB/s | 16 kB 00:00 2024-03-08T06:01:21.916 INFO:teuthology.orchestra.run.smithi040.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 167 kB/s | 272 kB 00:01 2024-03-08T06:01:22.108 INFO:teuthology.orchestra.run.smithi040.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 148 kB/s | 28 kB 00:00 2024-03-08T06:01:22.266 INFO:teuthology.orchestra.run.smithi040.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 100 kB/s | 90 kB 00:00 2024-03-08T06:01:22.308 INFO:teuthology.orchestra.run.smithi040.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 154 kB/s | 31 kB 00:00 2024-03-08T06:01:22.717 INFO:teuthology.orchestra.run.smithi040.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 145 kB/s | 59 kB 00:00 2024-03-08T06:01:22.892 INFO:teuthology.orchestra.run.smithi040.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 205 kB/s | 36 kB 00:00 2024-03-08T06:01:23.184 INFO:teuthology.orchestra.run.smithi040.stdout:(142/174): python3-typing-extensions-4.4.0-2.el 175 kB/s | 51 kB 00:00 2024-03-08T06:01:23.451 INFO:teuthology.orchestra.run.smithi040.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 159 kB/s | 188 kB 00:01 2024-03-08T06:01:23.534 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T06:01:23.574 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T06:01:24.064 INFO:teuthology.orchestra.run.smithi040.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 261 kB/s | 230 kB 00:00 2024-03-08T06:01:24.251 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T06:01:24.322 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T06:01:24.366 INFO:teuthology.orchestra.run.smithi040.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 98 kB/s | 90 kB 00:00 2024-03-08T06:01:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T06:01:24.495 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T06:01:24.495 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T06:01:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:24.536 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T06:01:24.608 INFO:teuthology.orchestra.run.smithi040.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 83 kB/s | 20 kB 00:00 2024-03-08T06:01:26.433 INFO:teuthology.orchestra.run.smithi040.stdout:(147/174): re2-20211101-3.el9.x86_64.rpm 105 kB/s | 192 kB 00:01 2024-03-08T06:01:26.601 INFO:teuthology.orchestra.run.smithi040.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 168 kB/s | 427 kB 00:02 2024-03-08T06:01:27.168 INFO:teuthology.orchestra.run.smithi040.stdout:(149/174): librados2-19.0.0-1670.g1fb5d106.el9. 6.0 MB/s | 3.4 MB 00:00 2024-03-08T06:01:27.535 INFO:teuthology.orchestra.run.smithi040.stdout:(150/174): librbd1-19.0.0-1670.g1fb5d106.el9.x8 8.6 MB/s | 3.1 MB 00:00 2024-03-08T06:01:27.585 INFO:teuthology.orchestra.run.smithi040.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-08T06:01:27.794 INFO:teuthology.orchestra.run.smithi040.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-03-08T06:01:27.886 INFO:teuthology.orchestra.run.smithi040.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-03-08T06:01:27.927 INFO:teuthology.orchestra.run.smithi040.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-08T06:01:27.978 INFO:teuthology.orchestra.run.smithi040.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-03-08T06:01:28.020 INFO:teuthology.orchestra.run.smithi040.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-08T06:01:28.062 INFO:teuthology.orchestra.run.smithi040.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 888 kB/s | 36 kB 00:00 2024-03-08T06:01:28.112 INFO:teuthology.orchestra.run.smithi040.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-03-08T06:01:28.179 INFO:teuthology.orchestra.run.smithi040.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.8 MB/s | 256 kB 00:00 2024-03-08T06:01:28.221 INFO:teuthology.orchestra.run.smithi040.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 564 kB/s | 23 kB 00:00 2024-03-08T06:01:28.280 INFO:teuthology.orchestra.run.smithi040.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-03-08T06:01:28.330 INFO:teuthology.orchestra.run.smithi040.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.1 MB/s | 155 kB 00:00 2024-03-08T06:01:28.380 INFO:teuthology.orchestra.run.smithi040.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.7 MB/s | 136 kB 00:00 2024-03-08T06:01:28.422 INFO:teuthology.orchestra.run.smithi040.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 64 kB 00:00 2024-03-08T06:01:28.464 INFO:teuthology.orchestra.run.smithi040.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 699 kB/s | 29 kB 00:00 2024-03-08T06:01:28.548 INFO:teuthology.orchestra.run.smithi040.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 15 MB/s | 1.2 MB 00:00 2024-03-08T06:01:28.656 INFO:teuthology.orchestra.run.smithi040.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 20 MB/s | 2.2 MB 00:00 2024-03-08T06:01:28.774 INFO:teuthology.orchestra.run.smithi040.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 20 MB/s | 2.3 MB 00:00 2024-03-08T06:01:28.832 INFO:teuthology.orchestra.run.smithi040.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 7.9 MB/s | 465 kB 00:00 2024-03-08T06:01:28.874 INFO:teuthology.orchestra.run.smithi040.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 756 kB/s | 31 kB 00:00 2024-03-08T06:01:28.949 INFO:teuthology.orchestra.run.smithi040.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 14 MB/s | 1.1 MB 00:00 2024-03-08T06:01:28.991 INFO:teuthology.orchestra.run.smithi040.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.8 MB/s | 74 kB 00:00 2024-03-08T06:01:33.175 INFO:teuthology.orchestra.run.smithi040.stdout:(173/174): libarrow-9.0.0-10.el9.x86_64.rpm 120 kB/s | 4.4 MB 00:37 2024-03-08T06:01:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T06:01:38.207 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-03-08T06:01:38.207 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-03-08T06:01:38.207 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-03-08T06:01:38.207 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-03-08T06:01:38.207 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-03-08T06:01:38.207 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-03-08T06:01:38.207 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-03-08T06:01:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:38.439 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T06:01:38.490 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T06:01:38.491 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:01:38.491 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T06:01:38.491 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:40.202 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T06:01:40.312 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T06:01:40.312 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:01:40.312 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T06:01:40.312 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T06:01:40.312 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T06:01:40.313 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:40.566 INFO:teuthology.orchestra.run.smithi040.stdout:(174/174): thrift-0.15.0-2.el9.x86_64.rpm 115 kB/s | 1.6 MB 00:14 2024-03-08T06:01:40.568 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T06:01:40.572 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-03-08T06:01:40.573 INFO:teuthology.orchestra.run.smithi040.stdout:Total 3.8 MB/s | 223 MB 00:58 2024-03-08T06:01:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T06:01:40.667 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:01:40.667 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T06:01:40.668 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T06:01:40.668 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T06:01:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:41.186 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T06:01:41.276 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T06:01:41.276 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:01:41.276 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T06:01:41.276 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T06:01:41.276 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T06:01:41.276 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:41.396 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T06:01:42.049 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-03-08T06:01:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T06:01:42.114 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T06:01:42.165 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 139/198 2024-03-08T06:01:42.273 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 140/198 2024-03-08T06:01:42.301 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T06:01:42.452 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-03-08T06:01:42.452 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-03-08T06:01:42.642 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T06:01:42.677 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T06:01:43.848 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T06:01:43.885 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T06:01:44.037 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T06:01:44.224 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 144/198 2024-03-08T06:01:44.262 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T06:01:44.354 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T06:01:44.354 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:01:44.355 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T06:01:44.355 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T06:01:44.355 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T06:01:44.355 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T06:01:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T06:01:44.544 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 147/198 2024-03-08T06:01:44.853 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-03-08T06:01:44.854 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-03-08T06:01:45.561 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T06:01:45.599 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T06:01:45.668 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T06:01:45.668 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:01:45.668 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T06:01:45.668 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T06:01:45.669 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T06:01:45.669 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:45.716 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T06:01:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T06:01:45.787 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:01:45.787 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T06:01:45.787 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T06:01:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T06:01:45.952 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:01:45.952 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T06:01:45.952 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T06:01:45.952 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T06:01:45.952 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T06:01:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T06:01:46.441 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:01:46.442 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T06:01:46.442 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T06:01:46.442 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T06:01:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:51.499 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T06:01:51.579 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 154/198 2024-03-08T06:01:51.622 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 155/198 2024-03-08T06:01:51.661 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T06:01:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 157/198 2024-03-08T06:01:52.121 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T06:01:52.165 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 159/198 2024-03-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T06:01:52.289 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T06:01:52.342 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T06:01:52.347 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-03-08T06:01:52.377 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 163/198 2024-03-08T06:01:52.440 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T06:01:52.451 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T06:01:52.488 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T06:01:52.509 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T06:01:52.534 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T06:01:52.579 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T06:01:52.594 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T06:01:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T06:01:52.707 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T06:01:52.747 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T06:01:52.937 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T06:01:53.000 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T06:01:53.036 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T06:01:53.132 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T06:01:53.552 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T06:01:53.600 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T06:01:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T06:01:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T06:01:53.819 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T06:01:53.878 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T06:01:53.948 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T06:01:54.079 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T06:01:54.122 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T06:01:54.195 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T06:01:54.494 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T06:01:54.547 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T06:01:54.573 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T06:01:54.723 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T06:01:54.757 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T06:01:54.848 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T06:01:54.893 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T06:01:54.926 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T06:01:54.991 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T06:01:55.307 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 170/198 2024-03-08T06:01:55.419 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T06:01:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T06:01:55.483 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T06:01:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T06:01:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T06:01:55.676 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T06:01:55.712 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T06:01:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T06:01:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T06:01:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T06:01:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T06:01:55.932 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T06:01:56.009 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T06:01:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T06:01:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T06:01:56.125 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T06:01:56.206 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T06:01:56.208 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T06:01:56.263 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T06:01:56.311 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T06:01:56.378 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T06:01:56.417 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T06:01:56.418 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T06:01:56.469 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T06:01:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T06:01:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T06:01:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T06:01:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T06:01:56.682 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T06:01:56.715 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T06:01:56.748 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T06:01:56.778 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T06:01:56.785 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T06:01:56.786 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T06:01:56.942 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T06:01:57.066 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T06:01:57.100 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T06:01:57.132 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T06:01:57.132 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T06:01:57.164 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T06:01:57.232 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T06:01:57.296 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T06:01:57.327 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T06:01:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T06:01:57.360 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T06:01:57.417 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T06:01:57.440 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T06:01:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T06:01:57.491 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T06:01:57.584 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T06:01:57.586 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T06:01:57.633 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T06:01:57.698 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T06:01:57.747 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T06:01:57.787 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T06:01:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T06:01:57.890 INFO:teuthology.orchestra.run.smithi053.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-08T06:01:57.890 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:01:57.908 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T06:01:57.941 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 39/198 2024-03-08T06:01:57.994 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 40/198 2024-03-08T06:01:58.030 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 41/198 2024-03-08T06:01:58.143 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T06:01:58.227 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T06:01:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T06:01:58.525 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T06:01:58.586 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T06:01:59.294 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T06:01:59.398 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T06:01:59.501 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T06:01:59.559 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T06:01:59.585 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T06:01:59.629 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T06:01:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T06:01:59.762 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 53/198 2024-03-08T06:01:59.800 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T06:02:00.224 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T06:02:00.416 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T06:02:00.507 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T06:02:00.578 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T06:02:00.790 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T06:02:00.835 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T06:02:00.912 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T06:02:01.007 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T06:02:01.042 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T06:02:01.082 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T06:02:01.149 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T06:02:01.200 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T06:02:01.287 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T06:02:01.359 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T06:02:01.564 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T06:02:01.623 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T06:02:01.662 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T06:02:01.722 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T06:02:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T06:02:01.909 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T06:02:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T06:02:01.991 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T06:02:02.016 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T06:02:02.184 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T06:02:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T06:02:02.509 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T06:02:02.564 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T06:02:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 1/198 2024-03-08T06:02:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2/198 2024-03-08T06:02:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 3/198 2024-03-08T06:02:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 4/198 2024-03-08T06:02:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 5/198 2024-03-08T06:02:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 6/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 7/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 8/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 9/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 13/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 14/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 15/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 17/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 18/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 19/198 2024-03-08T06:02:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 20/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 21/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 22/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 23/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 24/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 25/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 26/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 27/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 28/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 29/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 30/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 31/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 32/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 33/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 34/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 35/198 2024-03-08T06:02:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 36/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 37/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 38/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 39/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T06:02:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T06:02:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T06:02:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T06:02:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T06:02:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T06:02:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T06:02:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T06:02:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T06:02:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T06:02:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T06:02:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T06:02:02.613 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T06:02:02.688 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T06:02:02.940 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T06:02:02.997 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T06:02:03.171 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T06:02:03.219 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T06:02:03.260 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T06:02:03.314 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T06:02:03.365 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T06:02:03.415 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T06:02:03.483 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T06:02:03.542 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T06:02:03.587 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T06:02:03.720 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T06:02:03.775 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T06:02:03.818 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T06:02:03.885 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T06:02:03.930 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T06:02:04.065 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T06:02:04.174 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T06:02:04.234 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T06:02:04.269 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T06:02:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T06:02:04.498 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T06:02:04.561 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T06:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T06:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-03-08T06:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T06:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T06:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: jq-1.6-16.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T06:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T06:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T06:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T06:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T06:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T06:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T06:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T06:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T06:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T06:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T06:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T06:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T06:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-03-08T06:02:05.554 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T06:02:05.609 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T06:02:05.642 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T06:02:05.699 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T06:02:05.813 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T06:02:05.851 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T06:02:05.929 DEBUG:teuthology.parallel:result is None 2024-03-08T06:02:06.093 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T06:02:06.131 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T06:02:06.226 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T06:02:07.064 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T06:02:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T06:02:08.805 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T06:02:08.894 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T06:02:08.933 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T06:02:08.963 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T06:02:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T06:02:09.373 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T06:02:09.409 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T06:02:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T06:02:09.692 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 125/198 2024-03-08T06:02:09.750 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T06:02:10.307 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T06:02:10.338 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T06:02:10.438 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T06:02:10.481 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 129/198 2024-03-08T06:02:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T06:02:13.202 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T06:02:13.859 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T06:02:13.906 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T06:02:14.061 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T06:02:14.061 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T06:02:14.061 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T06:02:14.061 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:14.107 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T06:02:27.260 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T06:02:27.260 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2024-03-08T06:02:27.260 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2024-03-08T06:02:27.260 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2024-03-08T06:02:27.260 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2024-03-08T06:02:27.260 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2024-03-08T06:02:27.260 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2024-03-08T06:02:27.260 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2024-03-08T06:02:27.260 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:27.456 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T06:02:27.506 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T06:02:27.506 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:02:27.506 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T06:02:27.506 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:29.157 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T06:02:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T06:02:29.238 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:02:29.238 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T06:02:29.238 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T06:02:29.238 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T06:02:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:29.466 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T06:02:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T06:02:29.538 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:02:29.538 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T06:02:29.538 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T06:02:29.538 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T06:02:29.539 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:30.042 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T06:02:30.115 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T06:02:30.115 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:02:30.115 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T06:02:30.115 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T06:02:30.115 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T06:02:30.116 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:30.235 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T06:02:30.871 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T06:02:30.911 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T06:02:30.961 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 139/198 2024-03-08T06:02:31.069 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 140/198 2024-03-08T06:02:31.098 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T06:02:31.431 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T06:02:31.466 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T06:02:32.636 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T06:02:32.666 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T06:02:32.815 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T06:02:32.997 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 144/198 2024-03-08T06:02:33.026 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T06:02:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T06:02:33.103 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:02:33.103 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T06:02:33.103 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T06:02:33.103 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T06:02:33.103 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T06:02:33.200 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T06:02:33.295 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 147/198 2024-03-08T06:02:34.306 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T06:02:34.338 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T06:02:34.423 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T06:02:34.423 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:02:34.423 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T06:02:34.423 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T06:02:34.423 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T06:02:34.423 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:34.472 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T06:02:34.553 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T06:02:34.554 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:02:34.554 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T06:02:34.554 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:34.631 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T06:02:34.717 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T06:02:34.717 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:02:34.717 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T06:02:34.717 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T06:02:34.717 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T06:02:34.717 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:35.115 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T06:02:35.211 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T06:02:35.211 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T06:02:35.211 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T06:02:35.211 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T06:02:35.211 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T06:02:35.211 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:40.254 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T06:02:40.299 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 154/198 2024-03-08T06:02:40.335 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 155/198 2024-03-08T06:02:40.374 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T06:02:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 157/198 2024-03-08T06:02:40.785 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T06:02:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 159/198 2024-03-08T06:02:40.888 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T06:02:40.971 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T06:02:41.006 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T06:02:41.042 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 163/198 2024-03-08T06:02:41.079 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T06:02:41.122 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T06:02:41.211 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T06:02:41.320 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T06:02:41.362 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T06:02:42.307 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T06:02:42.351 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T06:02:42.413 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T06:02:43.821 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 170/198 2024-03-08T06:02:43.925 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T06:02:44.006 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T06:02:44.150 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T06:02:44.186 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T06:02:44.211 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T06:02:44.237 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T06:02:44.278 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T06:02:44.279 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T06:02:44.433 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T06:02:44.474 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T06:02:44.474 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T06:02:44.613 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T06:02:44.662 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T06:02:44.702 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T06:02:44.736 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T06:02:44.768 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T06:02:44.838 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T06:02:44.907 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T06:02:44.956 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T06:02:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T06:02:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T06:02:45.081 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T06:02:45.114 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T06:02:45.152 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T06:02:45.152 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T06:02:45.482 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T06:02:45.482 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T06:02:45.621 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T06:02:45.650 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T06:02:45.676 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T06:02:45.743 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T06:02:45.786 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T06:02:45.908 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T06:02:46.180 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T06:02:46.180 INFO:teuthology.orchestra.run.smithi040.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-08T06:02:46.180 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:50.987 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 1/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 3/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 4/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 5/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 6/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 7/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 8/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 9/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 13/198 2024-03-08T06:02:50.988 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 14/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 15/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 17/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 18/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 19/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 20/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 21/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 22/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 23/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 24/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 25/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 26/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 27/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 28/198 2024-03-08T06:02:50.989 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 29/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 30/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 31/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 32/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 33/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 34/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 35/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 36/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 37/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 38/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 39/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T06:02:50.990 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T06:02:50.991 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T06:02:50.992 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T06:02:50.993 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T06:02:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T06:02:50.995 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T06:02:50.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T06:02:50.997 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T06:02:50.998 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T06:02:50.999 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T06:02:51.000 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T06:02:51.001 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T06:02:51.001 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T06:02:51.001 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T06:02:51.001 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T06:02:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T06:02:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:53.457 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-03-08T06:02:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T06:02:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T06:02:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T06:02:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T06:02:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T06:02:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T06:02:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T06:02:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T06:02:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T06:02:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T06:02:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T06:02:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T06:02:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T06:02:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T06:02:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T06:02:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: jq-1.6-16.el9.x86_64 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T06:02:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T06:02:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T06:02:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T06:02:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T06:02:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T06:02:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T06:02:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T06:02:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T06:02:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T06:02:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T06:02:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T06:02:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T06:02:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T06:02:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T06:02:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T06:02:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T06:02:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T06:02:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T06:02:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T06:02:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T06:02:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T06:02:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T06:02:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T06:02:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T06:02:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T06:02:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T06:02:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T06:02:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T06:02:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T06:02:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:02:53.473 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-03-08T06:02:54.166 DEBUG:teuthology.parallel:result is None 2024-03-08T06:02:54.167 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T06:02:54.452 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T06:02:54.514 INFO:teuthology.orchestra.run.smithi040.stdout:19.0.0-1670.g1fb5d106.el9 2024-03-08T06:02:54.515 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1670.g1fb5d106.el9 2024-03-08T06:02:54.515 INFO:teuthology.task.install:The correct ceph version 19.0.0-1670.g1fb5d106 is installed. 2024-03-08T06:02:54.517 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T06:02:54.657 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T06:02:54.733 INFO:teuthology.orchestra.run.smithi053.stdout:19.0.0-1670.g1fb5d106.el9 2024-03-08T06:02:54.734 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1670.g1fb5d106.el9 2024-03-08T06:02:54.734 INFO:teuthology.task.install:The correct ceph version 19.0.0-1670.g1fb5d106 is installed. 2024-03-08T06:02:54.735 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-08T06:02:54.735 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:02:54.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T06:02:54.817 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:02:54.817 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T06:02:54.926 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-08T06:02:54.926 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:02:54.926 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T06:02:55.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T06:02:55.112 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:02:55.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T06:02:55.220 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T06:02:55.317 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-08T06:02:55.318 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:02:55.318 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T06:02:55.399 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T06:02:55.512 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:02:55.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T06:02:55.615 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T06:02:55.760 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-08T06:02:55.760 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:02:55.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T06:02:55.839 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T06:02:55.952 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:02:55.952 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T06:02:56.064 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T06:02:56.166 INFO:teuthology.run_tasks:Running task ceph... 2024-03-08T06:02:56.247 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-08T06:02:56.247 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2024-03-08T06:02:56.252 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /var/log/ceph 2024-03-08T06:02:56.360 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-08T06:02:56.360 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-08T06:02:56.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-08T06:02:56.467 INFO:tasks.ceph:Creating extra log directories... 2024-03-08T06:02:56.467 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-08T06:02:56.471 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-08T06:02:56.581 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-08T06:02:56.581 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-08T06:02:56.581 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-08T06:02:56.581 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:02:56.581 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-08T06:02:56.627 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-08T06:02:56.831 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-08T06:02:56.910 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:02:56.910 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-08T06:02:56.969 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-08T06:02:57.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-08T06:02:57.368 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-08T06:02:57.368 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'missing required features'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-08T06:02:57.368 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425', 'branch': 'main', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}', 'email': 'jcollin@redhat.com', 'first_in_suite': False, 'job_id': '7585425', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'missing required features'], 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-jcollin-test_per-client-perf-counters', 'sha1': 'de7a7e179fbee9e314fa702f8f6c09d121c5e1a7'}}, 'owner': 'scheduled_jcollin@teuthology', 'priority': 10, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7313, 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-jcollin-test_per-client-perf-counters', 'suite_path': '/home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/joscollin/ceph.git', 'suite_sha1': 'de7a7e179fbee9e314fa702f8f6c09d121c5e1a7', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE31JNxLJ5xIvqhIrwCXCY/HPljTsB8D0lzrE06ebM6bqC9FPeEn+hA8Jfmm1hUfxwY8fvklHKl+ayl8QGGNwVM=', 'smithi053.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgUVmyKe/dO23KvZNq7LUj18qPW9uMLNPo7GkfiWAQj+cwmNKxO+iImtt3ST8tLKKwIeAz0qcdYZJTcyo+gcoQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'missing required features'], 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_admin.TestLabeledPerfCounters']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-08_05:43:46', 'tube': 'smithi', 'user': 'jcollin', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-08T06:02:57.369 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-08T06:02:57.373 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-08T06:02:57.432 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2024-03-08T06:02:57.452 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/run/ceph 2024-03-08T06:02:57.552 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:02:57.552 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T06:02:57.589 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T06:02:57.589 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-03-08T06:02:57.673 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T06:02:57.674 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T06:02:57.674 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-08T06:02:57.674 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T06:02:57.674 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T06:02:57.674 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-03-08 06:02:46.657198627 +0000 2024-03-08T06:02:57.674 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-03-08 05:59:29.493769166 +0000 2024-03-08T06:02:57.674 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-03-08 05:59:29.493769166 +0000 2024-03-08T06:02:57.674 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-03-08 05:59:29.493769166 +0000 2024-03-08T06:02:57.674 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T06:02:57.783 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-03-08T06:02:57.783 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-03-08T06:02:57.783 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000465328 s, 1.1 MB/s 2024-03-08T06:02:57.786 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T06:02:57.825 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-03-08T06:02:57.902 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T06:02:57.902 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T06:02:57.902 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-08T06:02:57.902 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T06:02:57.902 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T06:02:57.902 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-03-08 06:02:46.658198614 +0000 2024-03-08T06:02:57.902 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-03-08 05:59:30.068761781 +0000 2024-03-08T06:02:57.903 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-03-08 05:59:30.068761781 +0000 2024-03-08T06:02:57.903 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-03-08 05:59:30.068761781 +0000 2024-03-08T06:02:57.903 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T06:02:58.013 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-03-08T06:02:58.013 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-03-08T06:02:58.013 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000293212 s, 1.7 MB/s 2024-03-08T06:02:58.016 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T06:02:58.059 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-03-08T06:02:58.137 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T06:02:58.137 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T06:02:58.137 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-08T06:02:58.137 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T06:02:58.137 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T06:02:58.137 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-03-08 06:02:46.659198601 +0000 2024-03-08T06:02:58.137 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-03-08 05:59:30.665754114 +0000 2024-03-08T06:02:58.138 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-03-08 05:59:30.665754114 +0000 2024-03-08T06:02:58.138 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-03-08 05:59:30.665754114 +0000 2024-03-08T06:02:58.138 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T06:02:58.250 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-03-08T06:02:58.250 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-03-08T06:02:58.250 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000261025 s, 2.0 MB/s 2024-03-08T06:02:58.253 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T06:02:58.293 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-03-08T06:02:58.387 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T06:02:58.387 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T06:02:58.387 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-08T06:02:58.388 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T06:02:58.388 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T06:02:58.388 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-03-08 06:02:46.659198601 +0000 2024-03-08T06:02:58.388 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-03-08 05:59:31.245746665 +0000 2024-03-08T06:02:58.388 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-03-08 05:59:31.245746665 +0000 2024-03-08T06:02:58.388 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-03-08 05:59:31.245746665 +0000 2024-03-08T06:02:58.388 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T06:02:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-03-08T06:02:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-03-08T06:02:58.476 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000325782 s, 1.6 MB/s 2024-03-08T06:02:58.481 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T06:02:58.536 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T06:02:58.536 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:02:58.536 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T06:02:58.597 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T06:02:58.597 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-03-08T06:02:58.654 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T06:02:58.654 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T06:02:58.654 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-08T06:02:58.654 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T06:02:58.654 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T06:02:58.654 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-03-08 06:01:58.383924603 +0000 2024-03-08T06:02:58.654 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-03-08 05:59:29.479688393 +0000 2024-03-08T06:02:58.654 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-03-08 05:59:29.480688385 +0000 2024-03-08T06:02:58.654 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-03-08 05:59:29.479688393 +0000 2024-03-08T06:02:58.655 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T06:02:58.747 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-03-08T06:02:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-03-08T06:02:58.748 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000233207 s, 2.2 MB/s 2024-03-08T06:02:58.752 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T06:02:58.811 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-03-08T06:02:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T06:02:58.864 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T06:02:58.864 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-08T06:02:58.864 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T06:02:58.864 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T06:02:58.865 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-03-08 06:01:58.384924594 +0000 2024-03-08T06:02:58.865 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-03-08 05:59:30.111683345 +0000 2024-03-08T06:02:58.865 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-03-08 05:59:30.111683345 +0000 2024-03-08T06:02:58.865 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-03-08 05:59:30.111683345 +0000 2024-03-08T06:02:58.865 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T06:02:58.961 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-03-08T06:02:58.961 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-03-08T06:02:58.961 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000259009 s, 2.0 MB/s 2024-03-08T06:02:58.965 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T06:02:59.026 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-03-08T06:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T06:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T06:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-08T06:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T06:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T06:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-03-08 06:01:58.384924594 +0000 2024-03-08T06:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-03-08 05:59:30.685678759 +0000 2024-03-08T06:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-03-08 05:59:30.685678759 +0000 2024-03-08T06:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-03-08 05:59:30.685678759 +0000 2024-03-08T06:02:59.080 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T06:02:59.173 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-03-08T06:02:59.173 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-03-08T06:02:59.173 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000261158 s, 2.0 MB/s 2024-03-08T06:02:59.178 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T06:02:59.237 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-03-08T06:02:59.289 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T06:02:59.289 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T06:02:59.289 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-08T06:02:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T06:02:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T06:02:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-03-08 06:01:58.386924576 +0000 2024-03-08T06:02:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-03-08 05:59:31.248674263 +0000 2024-03-08T06:02:59.290 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-03-08 05:59:31.248674263 +0000 2024-03-08T06:02:59.290 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-03-08 05:59:31.248674263 +0000 2024-03-08T06:02:59.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T06:02:59.375 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-03-08T06:02:59.375 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-03-08T06:02:59.375 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000218045 s, 2.3 MB/s 2024-03-08T06:02:59.378 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T06:02:59.436 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T06:02:59.437 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-08T06:02:59.437 INFO:tasks.ceph:Generating config... 2024-03-08T06:02:59.438 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-08T06:02:59.438 INFO:tasks.ceph:[client] debug client = 20 2024-03-08T06:02:59.438 INFO:tasks.ceph:[client] debug ms = 1 2024-03-08T06:02:59.438 INFO:tasks.ceph:[client] pid file = 2024-03-08T06:02:59.438 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-08T06:02:59.438 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-08T06:02:59.438 INFO:tasks.ceph:[global] lockdep = True 2024-03-08T06:02:59.438 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-08T06:02:59.438 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-08T06:02:59.438 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-08T06:02:59.438 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-08T06:02:59.438 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-08T06:02:59.439 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-08T06:02:59.439 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-08T06:02:59.439 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-08T06:02:59.439 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-08T06:02:59.439 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-08T06:02:59.440 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-08T06:02:59.440 INFO:tasks.ceph:Setting up mon.a... 2024-03-08T06:02:59.441 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-08T06:02:59.603 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2024-03-08T06:02:59.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-08T06:02:59.757 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-08T06:02:59.854 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.b', '172.21.15.53'), ('mon.c', '[v2:172.21.15.53:3301,v1:172.21.15.53:6790]')] 2024-03-08T06:02:59.855 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790]', 'lockdep': True, 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-08T06:02:59.856 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:02:59.856 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-08T06:02:59.909 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --add b 172.21.15.53 --addv c '[v2:172.21.15.53:3301,v1:172.21.15.53:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid c4e20909-a427-4e44-a3fb-93b068b9d1f6 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = quincy 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:fsid c4e20909-a427-4e44-a3fb-93b068b9d1f6 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2024-03-08T06:03:00.068609+0000 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-03-08T06:03:00.068609+0000 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 17 (quincy) 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] mon.c 2024-03-08T06:03:00.068 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-08T06:03:00.071 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-08T06:03:00.127 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c4e20909-a427-4e44-a3fb-93b068b9d1f6... 2024-03-08T06:03:00.128 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-08T06:03:00.133 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-08T06:03:00.295 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.296 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-08T06:03:00.297 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790]" 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: lockdep = True 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on skipped message = False 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = c4e20909-a427-4e44-a3fb-93b068b9d1f6 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.298 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug trim objects = true 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2024-03-08T06:03:00.299 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: bdev async discard = True 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: bdev enable discard = True 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore allocator = bitmap 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2024-03-08T06:03:00.300 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 1/20 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 1/20 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 4/10 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-03-08T06:03:00.301 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2024-03-08T06:03:00.302 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: mon op complaint time = 120 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2024-03-08T06:03:00.303 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout: client mount timeout = 600 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = "" 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-03-08T06:03:00.304 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2024-03-08T06:03:00.305 INFO:teuthology.orchestra.run.smithi040.stdout:[mds] 2024-03-08T06:03:00.305 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds = 20 2024-03-08T06:03:00.305 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds balancer = 20 2024-03-08T06:03:00.305 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-03-08T06:03:00.305 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug frag = True 2024-03-08T06:03:00.305 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug scatterstat = True 2024-03-08T06:03:00.305 INFO:teuthology.orchestra.run.smithi040.stdout: mds op complaint time = 180 2024-03-08T06:03:00.305 INFO:teuthology.orchestra.run.smithi040.stdout: mds verify scatter = True 2024-03-08T06:03:00.305 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-03-08T06:03:00.305 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-03-08T06:03:00.305 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-03-08T06:03:00.322 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-03-08T06:03:00.322 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2024-03-08T06:03:00.322 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-08T06:03:00.322 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2024-03-08T06:03:00.322 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.322 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2024-03-08T06:03:00.322 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.322 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2024-03-08T06:03:00.322 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.322 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2024-03-08T06:03:00.323 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2024-03-08T06:03:00.324 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.40,172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790]" 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: lockdep = True 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on skipped message = False 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = c4e20909-a427-4e44-a3fb-93b068b9d1f6 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: osd mclock profile = high_recovery_ops 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2024-03-08T06:03:00.325 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug trim objects = true 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2024-03-08T06:03:00.326 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: bdev async discard = True 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: bdev enable discard = True 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore allocator = bitmap 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 1/20 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 1/20 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 4/10 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-03-08T06:03:00.327 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2024-03-08T06:03:00.328 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2024-03-08T06:03:00.329 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = "" 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2024-03-08T06:03:00.330 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds balancer = 20 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = True 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = True 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-03-08T06:03:00.331 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-03-08T06:03:00.372 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-08T06:03:00.372 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-08T06:03:00.529 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-08T06:03:00.530 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:00.530 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-08T06:03:00.582 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:00.582 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-08T06:03:00.634 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:03:00.635 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:00.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-08T06:03:00.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-08T06:03:00.780 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:00.780 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:00.879 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:03:00.880 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:00.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-08T06:03:00.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-08T06:03:01.043 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:01.043 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:01.141 INFO:tasks.ceph:Setting up mon nodes... 2024-03-08T06:03:01.142 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-08T06:03:01.142 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-08T06:03:01.338 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-08T06:03:01.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-08T06:03:01.561 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-08T06:03:01.573 INFO:tasks.ceph:Setting up mds nodes... 2024-03-08T06:03:01.573 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-08T06:03:01.772 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-08T06:03:01.781 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-08T06:03:01.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-08T06:03:02.066 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-08T06:03:02.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-08T06:03:02.175 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-08T06:03:02.373 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-08T06:03:02.383 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-08T06:03:02.474 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-08T06:03:02.663 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-08T06:03:02.672 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-08T06:03:02.771 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-08T06:03:02.771 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-08T06:03:02.901 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-08T06:03:02.958 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-08T06:03:03.088 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-08T06:03:03.137 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-08T06:03:03.269 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-08T06:03:03.328 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-08T06:03:03.462 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-08T06:03:03.531 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-08T06:03:03.531 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-08T06:03:03.531 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-08T06:03:03.636 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T06:03:03.637 INFO:tasks.ceph:role: osd.0 2024-03-08T06:03:03.637 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:03:03.637 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-08T06:03:03.738 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T06:03:03.738 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T06:03:03.739 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T06:03:03.739 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T06:03:03.739 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T06:03:03.739 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-03-08T06:03:03.739 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T06:03:03.739 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T06:03:03.739 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T06:03:03.739 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T06:03:04.247 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-03-08T06:03:04.253 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-03-08T06:03:04.254 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-08T06:03:04.444 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-08T06:03:04.559 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-08T06:03:04.658 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T06:03:04.658 INFO:tasks.ceph:role: osd.1 2024-03-08T06:03:04.658 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:03:04.658 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-08T06:03:04.758 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T06:03:04.758 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T06:03:04.758 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T06:03:04.758 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T06:03:04.758 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T06:03:04.758 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-03-08T06:03:04.758 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T06:03:04.758 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T06:03:04.758 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T06:03:04.759 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T06:03:05.243 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-03-08T06:03:05.251 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-03-08T06:03:05.252 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-08T06:03:05.397 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-08T06:03:05.492 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-08T06:03:05.586 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T06:03:05.586 INFO:tasks.ceph:role: osd.2 2024-03-08T06:03:05.586 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:03:05.587 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-08T06:03:05.688 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T06:03:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T06:03:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T06:03:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T06:03:05.688 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T06:03:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-03-08T06:03:05.688 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T06:03:05.688 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T06:03:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T06:03:05.688 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T06:03:06.199 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-03-08T06:03:06.207 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-03-08T06:03:06.208 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-08T06:03:06.385 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-08T06:03:06.491 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-08T06:03:06.581 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T06:03:06.581 INFO:tasks.ceph:role: osd.3 2024-03-08T06:03:06.581 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:03:06.581 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-08T06:03:06.682 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T06:03:06.682 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T06:03:06.682 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T06:03:06.682 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T06:03:06.682 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T06:03:06.682 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-03-08T06:03:06.682 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T06:03:06.683 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T06:03:06.683 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T06:03:06.683 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T06:03:07.134 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-03-08T06:03:07.141 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-03-08T06:03:07.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-08T06:03:07.291 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-08T06:03:07.383 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:07.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:07.541+0000 7f418a00d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-08T06:03:07.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:07.546+0000 7f418a00d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-08T06:03:07.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:07.546+0000 7f418a00d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-08T06:03:07.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:07.546+0000 7f418a00d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-08T06:03:07.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:07.546+0000 7f418a00d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-08T06:03:07.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:07.551+0000 7f418a00d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-08T06:03:11.007 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-08T06:03:11.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:11.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:11.244+0000 7f3bf9e68640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-08T06:03:11.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:11.244+0000 7f3bf9e68640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-08T06:03:11.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:11.244+0000 7f3bf9e68640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-08T06:03:11.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:11.244+0000 7f3bf9e68640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-08T06:03:11.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:11.244+0000 7f3bf9e68640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-08T06:03:11.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:11.247+0000 7f3bf9e68640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-08T06:03:14.170 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-08T06:03:14.267 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:14.413+0000 7f1e28448640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-08T06:03:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:14.414+0000 7f1e28448640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-08T06:03:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:14.414+0000 7f1e28448640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-08T06:03:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:14.414+0000 7f1e28448640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-08T06:03:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:14.414+0000 7f1e28448640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-08T06:03:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:14.416+0000 7f1e28448640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-08T06:03:17.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-08T06:03:17.695 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:17.837+0000 7efd85866640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-08T06:03:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:17.838+0000 7efd85866640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-08T06:03:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:17.838+0000 7efd85866640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-08T06:03:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:17.838+0000 7efd85866640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-08T06:03:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:17.838+0000 7efd85866640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-08T06:03:17.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:17.840+0000 7efd85866640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-08T06:03:21.022 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-08T06:03:21.123 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-08T06:03:21.228 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T06:03:21.228 INFO:tasks.ceph:role: osd.4 2024-03-08T06:03:21.229 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:03:21.229 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-08T06:03:21.325 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T06:03:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T06:03:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T06:03:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T06:03:21.325 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T06:03:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-03-08T06:03:21.326 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T06:03:21.326 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T06:03:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T06:03:21.326 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T06:03:21.359 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-03-08T06:03:21.364 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-03-08T06:03:21.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-08T06:03:21.484 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-08T06:03:21.576 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-08T06:03:21.672 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T06:03:21.672 INFO:tasks.ceph:role: osd.5 2024-03-08T06:03:21.672 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:03:21.672 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-08T06:03:21.763 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T06:03:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T06:03:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T06:03:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T06:03:21.764 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T06:03:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-03-08T06:03:21.764 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T06:03:21.764 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T06:03:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T06:03:21.764 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T06:03:21.798 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-03-08T06:03:21.805 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-03-08T06:03:21.805 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-08T06:03:21.930 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-08T06:03:22.021 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-08T06:03:22.103 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T06:03:22.103 INFO:tasks.ceph:role: osd.6 2024-03-08T06:03:22.103 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:03:22.103 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-08T06:03:22.196 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T06:03:22.196 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T06:03:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T06:03:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T06:03:22.197 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T06:03:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-03-08T06:03:22.197 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T06:03:22.197 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T06:03:22.197 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T06:03:22.197 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T06:03:22.231 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-03-08T06:03:22.239 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-03-08T06:03:22.239 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-08T06:03:22.353 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-08T06:03:22.440 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-08T06:03:22.532 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T06:03:22.532 INFO:tasks.ceph:role: osd.7 2024-03-08T06:03:22.532 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:03:22.532 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-08T06:03:22.629 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T06:03:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T06:03:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T06:03:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T06:03:22.629 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T06:03:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-03-08T06:03:22.629 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T06:03:22.629 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T06:03:22.629 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T06:03:22.630 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T06:03:22.664 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-03-08T06:03:22.669 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-03-08T06:03:22.670 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-08T06:03:22.791 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-08T06:03:22.876 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:23.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:23.033+0000 7f5a3db47640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-08T06:03:23.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:23.034+0000 7f5a3db47640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-08T06:03:23.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:23.034+0000 7f5a3db47640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-08T06:03:23.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:23.034+0000 7f5a3db47640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-08T06:03:23.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:23.034+0000 7f5a3db47640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-08T06:03:23.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:23.037+0000 7f5a3db47640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-08T06:03:26.220 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-08T06:03:26.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:26.462 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:26.462+0000 7f14f2037640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-08T06:03:26.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:26.463+0000 7f14f2037640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-08T06:03:26.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:26.463+0000 7f14f2037640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-08T06:03:26.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:26.463+0000 7f14f2037640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-08T06:03:26.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:26.463+0000 7f14f2037640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-08T06:03:26.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:26.465+0000 7f14f2037640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-08T06:03:27.370 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:03:27.378 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:03:29.399 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-08T06:03:29.495 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:29.644+0000 7f241f102640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-08T06:03:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:29.644+0000 7f241f102640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-08T06:03:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:29.644+0000 7f241f102640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-08T06:03:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:29.644+0000 7f241f102640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-08T06:03:29.645 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:29.644+0000 7f241f102640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-08T06:03:29.647 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:29.647+0000 7f241f102640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-08T06:03:32.581 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-08T06:03:32.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:32.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:32.828+0000 7febab171640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-08T06:03:32.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:32.828+0000 7febab171640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-08T06:03:32.829 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:32.828+0000 7febab171640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-08T06:03:32.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:32.828+0000 7febab171640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-08T06:03:32.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:32.828+0000 7febab171640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-08T06:03:32.831 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:32.831+0000 7febab171640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-08T06:03:36.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-08T06:03:36.113 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-08T06:03:36.114 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:36.114 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-08T06:03:36.220 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:36.220 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-08T06:03:36.321 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:36.321 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-08T06:03:36.421 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:36.421 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-08T06:03:36.521 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:36.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-08T06:03:36.625 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:36.625 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-08T06:03:36.721 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:36.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-08T06:03:36.820 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:36.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-08T06:03:36.925 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:36.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-08T06:03:37.026 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:37.026 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-08T06:03:37.130 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:37.130 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-08T06:03:37.231 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:37.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-08T06:03:37.330 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:37.330 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-08T06:03:37.430 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:37.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-08T06:03:37.533 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:37.533 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-08T06:03:37.594 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:37.594 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-08T06:03:37.652 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:37.652 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-08T06:03:37.712 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:37.712 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-08T06:03:37.766 INFO:tasks.ceph:Adding keys to all mons... 2024-03-08T06:03:37.767 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-08T06:03:37.775 DEBUG:teuthology.orchestra.run.smithi053:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-08T06:03:37.868 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2024-03-08T06:03:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAVqupl2/Y1FBAACtDcILA/I2BmxOe83fcDXg== 2024-03-08T06:03:37.868 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-03-08T06:03:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAVqupl71wSLhAAS87lXSzQxwiWfqT/tiUZGw== 2024-03-08T06:03:37.868 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2024-03-08T06:03:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAWqupllRL/AxAAeGht9vBbsgddcoXPRfRMVQ== 2024-03-08T06:03:37.869 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2024-03-08T06:03:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAbquplz4pjIBAA3PPVSAKqCC9t+pxsnUE79Q== 2024-03-08T06:03:37.869 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2024-03-08T06:03:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAfquplnfWjDhAADHjcJSEAtOX7N6T7cIW3SA== 2024-03-08T06:03:37.869 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2024-03-08T06:03:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAiquplvTvCGBAA8t2oQZMt5Z+a5XE4qfS+Aw== 2024-03-08T06:03:37.869 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2024-03-08T06:03:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAlquplNvQCMhAArufflsQ9Qshoa5wxpB/xSg== 2024-03-08T06:03:37.869 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2024-03-08T06:03:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAVquplmyB7IRAA1QRz+cd43qLk8GikJZCAUg== 2024-03-08T06:03:37.870 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-03-08T06:03:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAWquplPVhRFhAACZ826lQeDP/KhF9bgKWZig== 2024-03-08T06:03:37.870 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2024-03-08T06:03:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAWquplI7SRJxAAQ6ID823EOUIGkGMLNwUKpA== 2024-03-08T06:03:37.870 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2024-03-08T06:03:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQArquplXMoPAhAApEEt2OjJynIXrPnGUY1+Pw== 2024-03-08T06:03:37.870 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2024-03-08T06:03:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAuqupl5aafGxAAPRDg67jn/tZbqWzaYctCqQ== 2024-03-08T06:03:37.870 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2024-03-08T06:03:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAxqupltOB2JhAAFw4z1NKoHJWPC+uRJPLdmQ== 2024-03-08T06:03:37.871 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2024-03-08T06:03:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA0quply35uMRAAi3/Dr7awTdLwXmaRpdFHdg== 2024-03-08T06:03:37.871 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-03-08T06:03:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAWquplEsjANRAAl87CIlR0lmsGJba4MA4mdw== 2024-03-08T06:03:37.871 INFO:teuthology.orchestra.run.smithi040.stdout:[client.2] 2024-03-08T06:03:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAXquplQzlPBRAAOkbXTAGJvuhC9xYAs+mrpQ== 2024-03-08T06:03:37.871 INFO:teuthology.orchestra.run.smithi040.stdout:[client.1] 2024-03-08T06:03:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAXquplxCwhEBAA2QmAZcA6M1tw4l8rvYrZpQ== 2024-03-08T06:03:37.871 INFO:teuthology.orchestra.run.smithi040.stdout:[client.3] 2024-03-08T06:03:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAXquplB26ZGxAAPhP8P4DU1uK7K7HCaYfBRQ== 2024-03-08T06:03:37.872 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.y] 2024-03-08T06:03:37.872 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAVqupl2/Y1FBAACtDcILA/I2BmxOe83fcDXg== 2024-03-08T06:03:37.872 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2024-03-08T06:03:37.872 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAVqupl71wSLhAAS87lXSzQxwiWfqT/tiUZGw== 2024-03-08T06:03:37.872 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2024-03-08T06:03:37.872 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAWqupllRL/AxAAeGht9vBbsgddcoXPRfRMVQ== 2024-03-08T06:03:37.872 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.0] 2024-03-08T06:03:37.873 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAbquplz4pjIBAA3PPVSAKqCC9t+pxsnUE79Q== 2024-03-08T06:03:37.873 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.1] 2024-03-08T06:03:37.873 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAfquplnfWjDhAADHjcJSEAtOX7N6T7cIW3SA== 2024-03-08T06:03:37.873 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.2] 2024-03-08T06:03:37.873 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAiquplvTvCGBAA8t2oQZMt5Z+a5XE4qfS+Aw== 2024-03-08T06:03:37.873 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.3] 2024-03-08T06:03:37.873 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAlquplNvQCMhAArufflsQ9Qshoa5wxpB/xSg== 2024-03-08T06:03:37.873 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.x] 2024-03-08T06:03:37.873 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAVquplmyB7IRAA1QRz+cd43qLk8GikJZCAUg== 2024-03-08T06:03:37.874 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2024-03-08T06:03:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAWquplPVhRFhAACZ826lQeDP/KhF9bgKWZig== 2024-03-08T06:03:37.874 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2024-03-08T06:03:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAWquplI7SRJxAAQ6ID823EOUIGkGMLNwUKpA== 2024-03-08T06:03:37.874 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.4] 2024-03-08T06:03:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQArquplXMoPAhAApEEt2OjJynIXrPnGUY1+Pw== 2024-03-08T06:03:37.874 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.5] 2024-03-08T06:03:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAuqupl5aafGxAAPRDg67jn/tZbqWzaYctCqQ== 2024-03-08T06:03:37.874 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.6] 2024-03-08T06:03:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAxqupltOB2JhAAFw4z1NKoHJWPC+uRJPLdmQ== 2024-03-08T06:03:37.874 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.7] 2024-03-08T06:03:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA0quply35uMRAAi3/Dr7awTdLwXmaRpdFHdg== 2024-03-08T06:03:37.875 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-03-08T06:03:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAWquplEsjANRAAl87CIlR0lmsGJba4MA4mdw== 2024-03-08T06:03:37.875 INFO:teuthology.orchestra.run.smithi053.stdout:[client.2] 2024-03-08T06:03:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAXquplQzlPBRAAOkbXTAGJvuhC9xYAs+mrpQ== 2024-03-08T06:03:37.875 INFO:teuthology.orchestra.run.smithi053.stdout:[client.1] 2024-03-08T06:03:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAXquplxCwhEBAA2QmAZcA6M1tw4l8rvYrZpQ== 2024-03-08T06:03:37.875 INFO:teuthology.orchestra.run.smithi053.stdout:[client.3] 2024-03-08T06:03:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAXquplB26ZGxAAPhP8P4DU1uK7K7HCaYfBRQ== 2024-03-08T06:03:37.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T06:03:37.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T06:03:38.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:03:38.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:03:38.202 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:03:38.208 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:03:38.360 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:38.366 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:38.515 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:38.520 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:38.670 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:38.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:38.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:38.833 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:38.987 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T06:03:38.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T06:03:39.154 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:03:39.162 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:03:39.315 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:03:39.321 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T06:03:39.478 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:39.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:39.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:39.642 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:39.793 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:39.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:39.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:39.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T06:03:40.110 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:03:40.118 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:03:40.276 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:03:40.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:03:40.434 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:03:40.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:03:40.601 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:03:40.609 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T06:03:40.762 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-08T06:03:40.762 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-08T06:03:40.862 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-08T06:03:41.028 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-08T06:03:41.114 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-08T06:03:41.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-08T06:03:41.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-08T06:03:41.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-08T06:03:41.555 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-08T06:03:41.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-08T06:03:41.815 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:41.823 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-08T06:03:41.877 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-08T06:03:41.877 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-08T06:03:41.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-08T06:03:41.885 INFO:tasks.ceph.mon.a:Started 2024-03-08T06:03:41.886 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-08T06:03:41.886 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-08T06:03:41.892 INFO:tasks.ceph.mon.b:Started 2024-03-08T06:03:41.893 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-08T06:03:41.893 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-08T06:03:41.899 INFO:tasks.ceph.mon.c:Started 2024-03-08T06:03:41.899 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-08T06:03:41.899 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-08T06:03:41.899 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-08T06:03:41.905 INFO:tasks.ceph.mgr.y:Started 2024-03-08T06:03:41.905 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-08T06:03:41.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-08T06:03:41.911 INFO:tasks.ceph.mgr.x:Started 2024-03-08T06:03:41.911 INFO:tasks.ceph:Setting crush tunables to default 2024-03-08T06:03:41.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2024-03-08T06:03:42.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.091+0000 7fe10cc00640 1 Processor -- start 2024-03-08T06:03:42.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.091+0000 7fe10cc00640 1 -- start start 2024-03-08T06:03:42.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.092+0000 7fe10cc00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 0x7fe108168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:42.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.092+0000 7fe10cc00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10815f390 con 0x7fe108072060 2024-03-08T06:03:42.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.092+0000 7fe10cc00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10806d820 con 0x7fe1081584a0 2024-03-08T06:03:42.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.092+0000 7fe10cc00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10806db00 con 0x7fe10815ed20 2024-03-08T06:03:42.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.093+0000 7fe106800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 0x7fe108168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:42.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.093+0000 7fe0fd400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe108072060 0x7fe10815e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48478/0 (socket says 172.21.15.40:48478) 2024-03-08T06:03:42.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.093+0000 7fe0fd400640 1 -- 172.21.15.40:0/2852723457 learned_addr learned my addr 172.21.15.40:0/2852723457 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:03:42.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.093+0000 7fe106800640 1 -- 172.21.15.40:0/2852723457 >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 msgr2=0x7fe108168e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-08T06:03:42.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.093+0000 7fe106800640 1 -- 172.21.15.40:0/2852723457 >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 msgr2=0x7fe108168e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-08T06:03:42.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.093+0000 7fe106800640 1 --2- 172.21.15.40:0/2852723457 >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 0x7fe108168e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T06:03:42.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.093+0000 7fe106800640 1 --2- 172.21.15.40:0/2852723457 >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 0x7fe108168e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T06:03:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.294+0000 7fe106800640 1 --2- 172.21.15.40:0/2852723457 >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 0x7fe108168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.294+0000 7fe106800640 1 -- 172.21.15.40:0/2852723457 >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 msgr2=0x7fe108168e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-08T06:03:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.294+0000 7fe106800640 1 -- 172.21.15.40:0/2852723457 >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 msgr2=0x7fe108168e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-08T06:03:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.294+0000 7fe106800640 1 --2- 172.21.15.40:0/2852723457 >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 0x7fe108168e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T06:03:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.294+0000 7fe106800640 1 --2- 172.21.15.40:0/2852723457 >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 0x7fe108168e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T06:03:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.343+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453447222 0 0) 0x7fe10806d820 con 0x7fe1081584a0 2024-03-08T06:03:42.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.343+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0ec003520 con 0x7fe1081584a0 2024-03-08T06:03:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.344+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849005556 0 0) 0x7fe10815f390 con 0x7fe108072060 2024-03-08T06:03:42.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.344+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10806d820 con 0x7fe108072060 2024-03-08T06:03:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.344+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 <== mon.1 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3544759948 0 0) 0x7fe0ec003520 con 0x7fe1081584a0 2024-03-08T06:03:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.344+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 msgr2=0x7fe108168e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T06:03:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.344+0000 7fe105400640 1 --2- 172.21.15.40:0/2852723457 >> v2:172.21.15.40:3300/0 conn(0x7fe10815ed20 0x7fe108168e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.344+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 >> v1:172.21.15.40:6789/0 conn(0x7fe108072060 legacy=0x7fe10815e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.344+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10806dde0 con 0x7fe1081584a0 2024-03-08T06:03:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.344+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 <== mon.1 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fe0f8001670 con 0x7fe1081584a0 2024-03-08T06:03:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.344+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 >> v1:172.21.15.53:6789/0 conn(0x7fe1081584a0 legacy=0x7fe108071950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.344+0000 7fe105400640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec003b90 0x7fe0ec003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:42.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.345+0000 7fe105400640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0ec004800 0x7fe0ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.345+0000 7fe105400640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0ec005290 0x7fe0ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.345+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10806db00 con 0x7fe0ec003b90 2024-03-08T06:03:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.345+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec00c1e0 con 0x7fe0ec005290 2024-03-08T06:03:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.345+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec00c3e0 con 0x7fe0ec004800 2024-03-08T06:03:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.346+0000 7fe105e00640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec003b90 0x7fe0ec003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:42.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.346+0000 7fe106800640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0ec005290 0x7fe0ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:42.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.346+0000 7fe0fd400640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0ec004800 0x7fe0ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.347+0000 7fe105e00640 1 -- 172.21.15.40:0/2852723457 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0ec004800 msgr2=0x7fe0ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.348+0000 7fe105e00640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0ec004800 0x7fe0ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.348+0000 7fe105e00640 1 -- 172.21.15.40:0/2852723457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0ec005290 msgr2=0x7fe0ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.348+0000 7fe105e00640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0ec005290 0x7fe0ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.348+0000 7fe105e00640 1 -- 172.21.15.40:0/2852723457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec003520 con 0x7fe0ec003b90 2024-03-08T06:03:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.348+0000 7fe106800640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0ec005290 0x7fe0ec00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:03:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.348+0000 7fe0fd400640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0ec004800 0x7fe0ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:03:42.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.348+0000 7fe106800640 1 -- 172.21.15.40:0/2852723457 reap_dead start 2024-03-08T06:03:42.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.351+0000 7fe105e00640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec003b90 0x7fe0ec003f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8007ed0 tx=0x7fe0ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eeb6722a5d4838d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:42.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.352+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8039070 con 0x7fe0ec003b90 2024-03-08T06:03:42.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.352+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f802c8d0 con 0x7fe0ec003b90 2024-03-08T06:03:42.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.352+0000 7fe105400640 1 -- 172.21.15.40:0/2852723457 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8007af0 con 0x7fe0ec003b90 2024-03-08T06:03:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.353+0000 7fe10cc00640 1 -- 172.21.15.40:0/2852723457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec003b90 msgr2=0x7fe0ec003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.353+0000 7fe10cc00640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec003b90 0x7fe0ec003f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8007ed0 tx=0x7fe0ec005830 comp rx=0 tx=0).stop 2024-03-08T06:03:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.353+0000 7fe10cc00640 1 -- 172.21.15.40:0/2852723457 shutdown_connections 2024-03-08T06:03:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.353+0000 7fe10cc00640 1 --2- 172.21.15.40:0/2852723457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec003b90 0x7fe0ec003f90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.353+0000 7fe10cc00640 1 -- 172.21.15.40:0/2852723457 >> 172.21.15.40:0/2852723457 conn(0x7fe10806ae30 msgr2=0x7fe10806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.353+0000 7fe10cc00640 1 -- 172.21.15.40:0/2852723457 shutdown_connections 2024-03-08T06:03:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.353+0000 7fe10cc00640 1 -- 172.21.15.40:0/2852723457 wait complete. 2024-03-08T06:03:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.356+0000 7fe10cc00640 1 Processor -- start 2024-03-08T06:03:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.356+0000 7fe10cc00640 1 -- start start 2024-03-08T06:03:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.358+0000 7fe10cc00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0ec003b90 0x7fe1081f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.358+0000 7fe10cc00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1080b25a0 0x7fe1080b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.358+0000 7fe10cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1081f7030 0x7fe1081f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.359+0000 7fe10cc00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10806e5e0 con 0x7fe1080b25a0 2024-03-08T06:03:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.359+0000 7fe10cc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10815e610 con 0x7fe1081f7030 2024-03-08T06:03:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.359+0000 7fe10cc00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080b0b30 con 0x7fe0ec003b90 2024-03-08T06:03:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.359+0000 7fe106800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1081f7030 0x7fe1081f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.359+0000 7fe106800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1081f7030 0x7fe1081f74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:35228/0 (socket says 172.21.15.40:35228) 2024-03-08T06:03:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.359+0000 7fe106800640 1 -- 172.21.15.40:0/1177154914 learned_addr learned my addr 172.21.15.40:0/1177154914 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.359+0000 7fe105e00640 1 --2- 172.21.15.40:0/1177154914 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0ec003b90 0x7fe1081f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:42.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.359+0000 7fe0fd400640 1 --2- 172.21.15.40:0/1177154914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1080b25a0 0x7fe1080b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:42.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.360+0000 7fe106800640 1 -- 172.21.15.40:0/1177154914 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0ec003b90 msgr2=0x7fe1081f0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:42.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.360+0000 7fe106800640 1 --2- 172.21.15.40:0/1177154914 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0ec003b90 0x7fe1081f0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.360+0000 7fe106800640 1 -- 172.21.15.40:0/1177154914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1080b25a0 msgr2=0x7fe1080b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.360+0000 7fe106800640 1 --2- 172.21.15.40:0/1177154914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1080b25a0 0x7fe1080b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.360+0000 7fe106800640 1 -- 172.21.15.40:0/1177154914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1080b0e10 con 0x7fe1081f7030 2024-03-08T06:03:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.360+0000 7fe106800640 1 --2- 172.21.15.40:0/1177154914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1081f7030 0x7fe1081f74b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe100003fd0 tx=0x7fe100007700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.361+0000 7fe0fea00640 1 -- 172.21.15.40:0/1177154914 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100004040 con 0x7fe1081f7030 2024-03-08T06:03:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.361+0000 7fe0fea00640 1 -- 172.21.15.40:0/1177154914 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10001c070 con 0x7fe1081f7030 2024-03-08T06:03:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.361+0000 7fe0fea00640 1 -- 172.21.15.40:0/1177154914 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000168a0 con 0x7fe1081f7030 2024-03-08T06:03:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.361+0000 7fe10cc00640 1 -- 172.21.15.40:0/1177154914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1080b10f0 con 0x7fe1081f7030 2024-03-08T06:03:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.361+0000 7fe10cc00640 1 -- 172.21.15.40:0/1177154914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1081f4f80 con 0x7fe1081f7030 2024-03-08T06:03:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.361+0000 7fe0fea00640 1 -- 172.21.15.40:0/1177154914 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe100016a40 con 0x7fe1081f7030 2024-03-08T06:03:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.362+0000 7fe0fea00640 1 -- 172.21.15.40:0/1177154914 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe1000203c0 con 0x7fe1081f7030 2024-03-08T06:03:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.363+0000 7fe10cc00640 1 -- 172.21.15.40:0/1177154914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1081f5180 con 0x7fe1081f7030 2024-03-08T06:03:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.366+0000 7fe0fea00640 1 -- 172.21.15.40:0/1177154914 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe1081f5180 con 0x7fe1081f7030 2024-03-08T06:03:42.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.442+0000 7fe10cc00640 1 -- 172.21.15.40:0/1177154914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe108067cb0 con 0x7fe1081f7030 2024-03-08T06:03:42.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.450+0000 7fe0fea00640 1 -- 172.21.15.40:0/1177154914 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe100021460 con 0x7fe1081f7030 2024-03-08T06:03:42.450 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2024-03-08T06:03:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.451+0000 7fe10cc00640 1 -- 172.21.15.40:0/1177154914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1081f7030 msgr2=0x7fe1081f74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.451+0000 7fe10cc00640 1 --2- 172.21.15.40:0/1177154914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1081f7030 0x7fe1081f74b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe100003fd0 tx=0x7fe100007700 comp rx=0 tx=0).stop 2024-03-08T06:03:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.452+0000 7fe10cc00640 1 -- 172.21.15.40:0/1177154914 shutdown_connections 2024-03-08T06:03:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.452+0000 7fe10cc00640 1 --2- 172.21.15.40:0/1177154914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1081f7030 0x7fe1081f74b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.452+0000 7fe10cc00640 1 --2- 172.21.15.40:0/1177154914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1080b25a0 0x7fe1080b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.452+0000 7fe10cc00640 1 --2- 172.21.15.40:0/1177154914 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0ec003b90 0x7fe1081f0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:42.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.452+0000 7fe10cc00640 1 -- 172.21.15.40:0/1177154914 >> 172.21.15.40:0/1177154914 conn(0x7fe10806ae30 msgr2=0x7fe108145800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:42.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.452+0000 7fe10cc00640 1 -- 172.21.15.40:0/1177154914 shutdown_connections 2024-03-08T06:03:42.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:42.452+0000 7fe10cc00640 1 -- 172.21.15.40:0/1177154914 wait complete. 2024-03-08T06:03:42.469 INFO:tasks.ceph:check_enable_crimson: False 2024-03-08T06:03:42.469 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-08T06:03:42.469 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:42.469 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-08T06:03:42.547 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:42.547 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-08T06:03:42.681 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:42.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-08T06:03:42.779 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:03:42.779 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-08T06:03:42.864 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:42.864 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-08T06:03:42.930 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:42.931+0000 7f5845a42040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T06:03:42.957 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:42.957 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-08T06:03:42.972 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:42.972+0000 7fdee322b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T06:03:42.996 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:42.996+0000 7f5845a42040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T06:03:43.047 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:43.047 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-08T06:03:43.047 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:43.046+0000 7fdee322b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T06:03:43.178 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:03:43.178 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-08T06:03:43.264 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new ea92b754-fbb9-461c-8332-5343a83796c3 0 2024-03-08T06:03:43.507 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.506+0000 7f27e3000640 1 Processor -- start 2024-03-08T06:03:43.508 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.507+0000 7f27e3000640 1 -- start start 2024-03-08T06:03:43.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.508+0000 7f27e3000640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27dc15de70 0x7f27dc0b5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:43.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.508+0000 7f27e3000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27dc15e250 con 0x7f27dc0d2b20 2024-03-08T06:03:43.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.508+0000 7f27e3000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27dc0cf1f0 con 0x7f27dc0cec60 2024-03-08T06:03:43.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.508+0000 7f27e3000640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27dc0d1d60 con 0x7f27dc15de70 2024-03-08T06:03:43.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.509+0000 7f27dbe00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27dc15de70 0x7f27dc0b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:43.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.509+0000 7f27dbe00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27dc15de70 0x7f27dc0b5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39334/0 (socket says 172.21.15.53:39334) 2024-03-08T06:03:43.510 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.509+0000 7f27dbe00640 1 -- 172.21.15.53:0/1195661974 learned_addr learned my addr 172.21.15.53:0/1195661974 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-08T06:03:43.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.509+0000 7f27e1000640 1 --1- 172.21.15.53:0/1195661974 >> v1:172.21.15.53:6789/0 conn(0x7f27dc0cec60 0x7f27dc0b6d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58294/0 (socket says 172.21.15.53:58294) 2024-03-08T06:03:43.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.509+0000 7f27db400640 1 -- 172.21.15.53:0/1195661974 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181364801 0 0) 0x7f27dc0cf1f0 con 0x7f27dc0cec60 2024-03-08T06:03:43.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.509+0000 7f27dbe00640 1 -- 172.21.15.53:0/1195661974 >> v1:172.21.15.53:6789/0 conn(0x7f27dc0cec60 legacy=0x7f27dc0b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:43.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.509+0000 7f27dbe00640 1 -- 172.21.15.53:0/1195661974 >> v1:172.21.15.40:6789/0 conn(0x7f27dc0d2b20 legacy=0x7f27dc0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:43.511 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.509+0000 7f27dbe00640 1 -- 172.21.15.53:0/1195661974 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc0d2040 con 0x7f27dc15de70 2024-03-08T06:03:43.512 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.510+0000 7f27dbe00640 1 --2- 172.21.15.53:0/1195661974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27dc15de70 0x7f27dc0b5770 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0098a0 tx=0x7f27cc02ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52c52dcf5371c7fb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.511+0000 7f27db400640 1 -- 172.21.15.53:0/1195661974 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc02f830 con 0x7f27dc15de70 2024-03-08T06:03:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.511+0000 7f27db400640 1 -- 172.21.15.53:0/1195661974 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27cc02f9d0 con 0x7f27dc15de70 2024-03-08T06:03:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.511+0000 7f27db400640 1 -- 172.21.15.53:0/1195661974 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc02fda0 con 0x7f27dc15de70 2024-03-08T06:03:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.511+0000 7f27e3000640 1 -- 172.21.15.53:0/1195661974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27dc15de70 msgr2=0x7f27dc0b5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.511+0000 7f27e3000640 1 --2- 172.21.15.53:0/1195661974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27dc15de70 0x7f27dc0b5770 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0098a0 tx=0x7f27cc02ba10 comp rx=0 tx=0).stop 2024-03-08T06:03:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.511+0000 7f27e3000640 1 -- 172.21.15.53:0/1195661974 shutdown_connections 2024-03-08T06:03:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.511+0000 7f27e3000640 1 --2- 172.21.15.53:0/1195661974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27dc15de70 0x7f27dc0b5770 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.511+0000 7f27e3000640 1 -- 172.21.15.53:0/1195661974 >> 172.21.15.53:0/1195661974 conn(0x7f27dc067240 msgr2=0x7f27dc0d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.511+0000 7f27e3000640 1 -- 172.21.15.53:0/1195661974 shutdown_connections 2024-03-08T06:03:43.513 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.511+0000 7f27e3000640 1 -- 172.21.15.53:0/1195661974 wait complete. 2024-03-08T06:03:43.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.513+0000 7f27e3000640 1 Processor -- start 2024-03-08T06:03:43.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.513+0000 7f27e3000640 1 -- start start 2024-03-08T06:03:43.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.514+0000 7f27e3000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27dc0cec60 0x7f27dc0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:43.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.514+0000 7f27e3000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f27dc0d2b20 0x7f27dc0c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:43.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.514+0000 7f27e3000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27dc15de70 0x7f27dc0c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:43.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.514+0000 7f27e3000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc0b5cb0 con 0x7f27dc15de70 2024-03-08T06:03:43.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.514+0000 7f27e3000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc0d2600 con 0x7f27dc0cec60 2024-03-08T06:03:43.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.514+0000 7f27e3000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc0b5e30 con 0x7f27dc0d2b20 2024-03-08T06:03:43.517 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.514+0000 7f27dbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27dc0cec60 0x7f27dc0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.514+0000 7f27dbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27dc0cec60 0x7f27dc0cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39348/0 (socket says 172.21.15.53:39348) 2024-03-08T06:03:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.514+0000 7f27dbe00640 1 -- 172.21.15.53:0/1694280639 learned_addr learned my addr 172.21.15.53:0/1694280639 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-08T06:03:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.514+0000 7f27d3e00640 1 --2- 172.21.15.53:0/1694280639 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f27dc0d2b20 0x7f27dc0c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.515+0000 7f27e1000640 1 --2- 172.21.15.53:0/1694280639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27dc15de70 0x7f27dc0c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.515+0000 7f27dbe00640 1 -- 172.21.15.53:0/1694280639 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f27dc0d2b20 msgr2=0x7f27dc0c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.515+0000 7f27dbe00640 1 --2- 172.21.15.53:0/1694280639 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f27dc0d2b20 0x7f27dc0c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.515+0000 7f27dbe00640 1 -- 172.21.15.53:0/1694280639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27dc15de70 msgr2=0x7f27dc0c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.515+0000 7f27dbe00640 1 --2- 172.21.15.53:0/1694280639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27dc15de70 0x7f27dc0c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.515+0000 7f27dbe00640 1 -- 172.21.15.53:0/1694280639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc0c8850 con 0x7f27dc0cec60 2024-03-08T06:03:43.518 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.515+0000 7f27dbe00640 1 --2- 172.21.15.53:0/1694280639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27dc0cec60 0x7f27dc0cadc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0098a0 tx=0x7f27cc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.515+0000 7f27d8c00640 1 -- 172.21.15.53:0/1694280639 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc02f7e0 con 0x7f27dc0cec60 2024-03-08T06:03:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.515+0000 7f27d8c00640 1 -- 172.21.15.53:0/1694280639 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27cc046070 con 0x7f27dc0cec60 2024-03-08T06:03:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.515+0000 7f27d8c00640 1 -- 172.21.15.53:0/1694280639 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc041640 con 0x7f27dc0cec60 2024-03-08T06:03:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.515+0000 7f27e3000640 1 -- 172.21.15.53:0/1694280639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27dc14d0f0 con 0x7f27dc0cec60 2024-03-08T06:03:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.516+0000 7f27e3000640 1 -- 172.21.15.53:0/1694280639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc14d6b0 con 0x7f27dc0cec60 2024-03-08T06:03:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.516+0000 7f27d8c00640 1 -- 172.21.15.53:0/1694280639 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f27cc040070 con 0x7f27dc0cec60 2024-03-08T06:03:43.519 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.516+0000 7f27d8c00640 1 -- 172.21.15.53:0/1694280639 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f27cc04b5a0 con 0x7f27dc0cec60 2024-03-08T06:03:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.517+0000 7f27e3000640 1 -- 172.21.15.53:0/1694280639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc067cb0 con 0x7f27dc0cec60 2024-03-08T06:03:43.520 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.519+0000 7f27d8c00640 1 -- 172.21.15.53:0/1694280639 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f27cc033030 con 0x7f27dc0cec60 2024-03-08T06:03:43.599 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.598+0000 7f27e3000640 1 -- 172.21.15.53:0/1694280639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ea92b754-fbb9-461c-8332-5343a83796c3", "id": 0} v 0) v1 -- 0x7f27dc0c7210 con 0x7f27dc0cec60 2024-03-08T06:03:43.607 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.606+0000 7f27d8c00640 1 -- 172.21.15.53:0/1694280639 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ea92b754-fbb9-461c-8332-5343a83796c3", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f27cc02f980 con 0x7f27dc0cec60 2024-03-08T06:03:43.607 INFO:teuthology.orchestra.run.smithi053.stdout:0 2024-03-08T06:03:43.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.608+0000 7f27e3000640 1 -- 172.21.15.53:0/1694280639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27dc0cec60 msgr2=0x7f27dc0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:43.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.608+0000 7f27e3000640 1 --2- 172.21.15.53:0/1694280639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27dc0cec60 0x7f27dc0cadc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0098a0 tx=0x7f27cc004730 comp rx=0 tx=0).stop 2024-03-08T06:03:43.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.608+0000 7f27e3000640 1 -- 172.21.15.53:0/1694280639 shutdown_connections 2024-03-08T06:03:43.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.608+0000 7f27e3000640 1 --2- 172.21.15.53:0/1694280639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27dc15de70 0x7f27dc0c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.608+0000 7f27e3000640 1 --2- 172.21.15.53:0/1694280639 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f27dc0d2b20 0x7f27dc0c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.608+0000 7f27e3000640 1 --2- 172.21.15.53:0/1694280639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27dc0cec60 0x7f27dc0cadc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.608+0000 7f27e3000640 1 -- 172.21.15.53:0/1694280639 >> 172.21.15.53:0/1694280639 conn(0x7f27dc067240 msgr2=0x7f27dc0d2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:43.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.608+0000 7f27e3000640 1 -- 172.21.15.53:0/1694280639 shutdown_connections 2024-03-08T06:03:43.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.608+0000 7f27e3000640 1 -- 172.21.15.53:0/1694280639 wait complete. 2024-03-08T06:03:43.615 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:43.615+0000 7f5845a42040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T06:03:43.627 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 022bb897-a67f-4555-a1be-0d81a4e306d4 1 2024-03-08T06:03:43.683 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:43.682+0000 7f5845a42040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T06:03:43.706 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:43.705+0000 7fdee322b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T06:03:43.736 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:43.736+0000 7f5845a42040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T06:03:43.775 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:43.775+0000 7fdee322b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T06:03:43.802 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:43.802+0000 7f5845a42040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T06:03:43.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.815+0000 7f5683e00640 1 Processor -- start 2024-03-08T06:03:43.816 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.815+0000 7f5683e00640 1 -- start start 2024-03-08T06:03:43.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.816+0000 7f5683e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5684161410 0x7f56840b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:43.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f5683e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56840d3500 0x7f56840b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:43.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f5683e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 0x7f56840b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:43.818 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f5683e00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56840d1060 con 0x7f5684161410 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f5683e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56841617f0 con 0x7f56840d3500 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f5683e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56840d12b0 con 0x7f56840b6c90 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f567be00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56840d3500 0x7f56840b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f567be00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56840d3500 0x7f56840b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54202/0 (socket says 172.21.15.53:54202) 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f567be00640 1 -- 172.21.15.53:0/1580541578 learned_addr learned my addr 172.21.15.53:0/1580541578 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f5683400640 1 --2- 172.21.15.53:0/1580541578 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 0x7f56840b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f5682a00640 1 --2- 172.21.15.53:0/1580541578 >> v2:172.21.15.40:3300/0 conn(0x7f5684161410 0x7f56840b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f567be00640 1 -- 172.21.15.53:0/1580541578 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 msgr2=0x7f56840b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f567be00640 1 --2- 172.21.15.53:0/1580541578 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 0x7f56840b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f567be00640 1 -- 172.21.15.53:0/1580541578 >> v2:172.21.15.40:3300/0 conn(0x7f5684161410 msgr2=0x7f56840b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f567be00640 1 --2- 172.21.15.53:0/1580541578 >> v2:172.21.15.40:3300/0 conn(0x7f5684161410 0x7f56840b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f567be00640 1 -- 172.21.15.53:0/1580541578 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56840cebd0 con 0x7f56840d3500 2024-03-08T06:03:43.819 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.817+0000 7f5683400640 1 --2- 172.21.15.53:0/1580541578 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 0x7f56840b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:03:43.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.819+0000 7f567be00640 1 --2- 172.21.15.53:0/1580541578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56840d3500 0x7f56840b5570 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f56840d0540 tx=0x7f566c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a0d186d9b76864 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:43.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.819+0000 7f5682000640 1 -- 172.21.15.53:0/1580541578 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c035020 con 0x7f56840d3500 2024-03-08T06:03:43.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.819+0000 7f5682000640 1 -- 172.21.15.53:0/1580541578 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c031870 con 0x7f56840d3500 2024-03-08T06:03:43.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.819+0000 7f5682000640 1 -- 172.21.15.53:0/1580541578 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c031c40 con 0x7f56840d3500 2024-03-08T06:03:43.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.819+0000 7f5683e00640 1 -- 172.21.15.53:0/1580541578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56840d3500 msgr2=0x7f56840b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:43.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.819+0000 7f5683e00640 1 --2- 172.21.15.53:0/1580541578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56840d3500 0x7f56840b5570 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f56840d0540 tx=0x7f566c02b910 comp rx=0 tx=0).stop 2024-03-08T06:03:43.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.820+0000 7f5683e00640 1 -- 172.21.15.53:0/1580541578 shutdown_connections 2024-03-08T06:03:43.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.820+0000 7f5683e00640 1 --2- 172.21.15.53:0/1580541578 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 0x7f56840b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.820+0000 7f5683e00640 1 --2- 172.21.15.53:0/1580541578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56840d3500 0x7f56840b5570 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.821 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.820+0000 7f5683e00640 1 --2- 172.21.15.53:0/1580541578 >> v2:172.21.15.40:3300/0 conn(0x7f5684161410 0x7f56840b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.820+0000 7f5683e00640 1 -- 172.21.15.53:0/1580541578 >> 172.21.15.53:0/1580541578 conn(0x7f568406c740 msgr2=0x7f568406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:43.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.820+0000 7f5683e00640 1 -- 172.21.15.53:0/1580541578 shutdown_connections 2024-03-08T06:03:43.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.820+0000 7f5683e00640 1 -- 172.21.15.53:0/1580541578 wait complete. 2024-03-08T06:03:43.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.821+0000 7f5683e00640 1 Processor -- start 2024-03-08T06:03:43.822 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.821+0000 7f5683e00640 1 -- start start 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.822+0000 7f5683e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 0x7f56840cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.822+0000 7f5683e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684161410 0x7f568415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.822+0000 7f5683e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684162070 0x7f5684151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.822+0000 7f5683e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56840b2f60 con 0x7f5684162070 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.822+0000 7f5683e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56840b2de0 con 0x7f5684161410 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.822+0000 7f5683e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5684161a70 con 0x7f56840b6c90 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f5682a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 0x7f56840cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f567be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684161410 0x7f568415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f567be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684161410 0x7f568415be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54216/0 (socket says 172.21.15.53:54216) 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f5682a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 0x7f56840cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:53278/0 (socket says 172.21.15.53:53278) 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f567be00640 1 -- 172.21.15.53:0/3241217764 learned_addr learned my addr 172.21.15.53:0/3241217764 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-08T06:03:43.825 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f5683400640 1 --2- 172.21.15.53:0/3241217764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684162070 0x7f5684151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f567be00640 1 -- 172.21.15.53:0/3241217764 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 msgr2=0x7f56840cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f567be00640 1 --2- 172.21.15.53:0/3241217764 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 0x7f56840cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f567be00640 1 -- 172.21.15.53:0/3241217764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684162070 msgr2=0x7f5684151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f567be00640 1 --2- 172.21.15.53:0/3241217764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684162070 0x7f5684151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f567be00640 1 -- 172.21.15.53:0/3241217764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568415c340 con 0x7f5684161410 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f5682a00640 1 --2- 172.21.15.53:0/3241217764 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 0x7f56840cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.823+0000 7f567be00640 1 --2- 172.21.15.53:0/3241217764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684161410 0x7f568415be00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f566c000c00 tx=0x7f566c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.824+0000 7f567aa00640 1 -- 172.21.15.53:0/3241217764 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c035040 con 0x7f5684161410 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.824+0000 7f567aa00640 1 -- 172.21.15.53:0/3241217764 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c030d00 con 0x7f5684161410 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.824+0000 7f5683e00640 1 -- 172.21.15.53:0/3241217764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56840cb1a0 con 0x7f5684161410 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.824+0000 7f567aa00640 1 -- 172.21.15.53:0/3241217764 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c0456a0 con 0x7f5684161410 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.824+0000 7f5683e00640 1 -- 172.21.15.53:0/3241217764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56840cb760 con 0x7f5684161410 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.824+0000 7f567aa00640 1 -- 172.21.15.53:0/3241217764 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f566c038020 con 0x7f5684161410 2024-03-08T06:03:43.826 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.824+0000 7f567aa00640 1 -- 172.21.15.53:0/3241217764 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f566c03a880 con 0x7f5684161410 2024-03-08T06:03:43.827 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.825+0000 7f5683e00640 1 -- 172.21.15.53:0/3241217764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5684075100 con 0x7f5684161410 2024-03-08T06:03:43.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.830+0000 7f567aa00640 1 -- 172.21.15.53:0/3241217764 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f566c053080 con 0x7f5684161410 2024-03-08T06:03:43.833 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:43.833+0000 7fdee322b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T06:03:43.855 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:43.855+0000 7f5845a42040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T06:03:43.901 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:43.901+0000 7fdee322b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T06:03:43.910 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.909+0000 7f5683e00640 1 -- 172.21.15.53:0/3241217764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd new", "uuid": "022bb897-a67f-4555-a1be-0d81a4e306d4", "id": 1} v 0) v1 -- 0x7f5684160b00 con 0x7f5684161410 2024-03-08T06:03:43.917 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.917+0000 7f567aa00640 1 -- 172.21.15.53:0/3241217764 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "022bb897-a67f-4555-a1be-0d81a4e306d4", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f566c03a420 con 0x7f5684161410 2024-03-08T06:03:43.917 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-03-08T06:03:43.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.918+0000 7f5683e00640 1 -- 172.21.15.53:0/3241217764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684161410 msgr2=0x7f568415be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:43.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.918+0000 7f5683e00640 1 --2- 172.21.15.53:0/3241217764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684161410 0x7f568415be00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f566c000c00 tx=0x7f566c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.918+0000 7f5683e00640 1 -- 172.21.15.53:0/3241217764 shutdown_connections 2024-03-08T06:03:43.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.918+0000 7f5683e00640 1 --2- 172.21.15.53:0/3241217764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684162070 0x7f5684151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.918+0000 7f5683e00640 1 --2- 172.21.15.53:0/3241217764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5684161410 0x7f568415be00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.918+0000 7f5683e00640 1 --2- 172.21.15.53:0/3241217764 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56840b6c90 0x7f56840cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:43.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.918+0000 7f5683e00640 1 -- 172.21.15.53:0/3241217764 >> 172.21.15.53:0/3241217764 conn(0x7f568406c740 msgr2=0x7f56840d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:43.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.918+0000 7f5683e00640 1 -- 172.21.15.53:0/3241217764 shutdown_connections 2024-03-08T06:03:43.919 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:43.918+0000 7f5683e00640 1 -- 172.21.15.53:0/3241217764 wait complete. 2024-03-08T06:03:43.935 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 9296f075-c6f9-416e-99dd-13e3db81e1be 2 2024-03-08T06:03:43.957 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:43.957+0000 7fdee322b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T06:03:43.992 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:43.992+0000 7f5845a42040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T06:03:44.076 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:44.077+0000 7f5845a42040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T06:03:44.099 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:44.098+0000 7fdee322b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T06:03:44.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.128+0000 7f6cb2000640 1 Processor -- start 2024-03-08T06:03:44.128 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.128+0000 7f6cb2000640 1 -- start start 2024-03-08T06:03:44.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.129+0000 7f6cb2000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6cac15dcc0 0x7f6cac0d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.129+0000 7f6cb2000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cac0d25d0 con 0x7f6cac0d3670 2024-03-08T06:03:44.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.129+0000 7f6cb2000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cac0d28b0 con 0x7f6cac06c9a0 2024-03-08T06:03:44.130 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.129+0000 7f6cb2000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac0d2b90 con 0x7f6cac15dcc0 2024-03-08T06:03:44.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.130+0000 7f6cb0c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6cac15dcc0 0x7f6cac0d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.130+0000 7f6cb1600640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6cac06c9a0 0x7f6cac0b6c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57960/0 (socket says 172.21.15.53:57960) 2024-03-08T06:03:44.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.130+0000 7f6cb1600640 1 -- 172.21.15.53:0/1648678133 learned_addr learned my addr 172.21.15.53:0/1648678133 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-03-08T06:03:44.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.130+0000 7f6cab400640 1 -- 172.21.15.53:0/1648678133 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147951088 0 0) 0x7f6cac0d28b0 con 0x7f6cac06c9a0 2024-03-08T06:03:44.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.130+0000 7f6cb0c00640 1 -- 172.21.15.53:0/1648678133 >> v1:172.21.15.53:6789/0 conn(0x7f6cac06c9a0 legacy=0x7f6cac0b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.130+0000 7f6cb0c00640 1 -- 172.21.15.53:0/1648678133 >> v1:172.21.15.40:6789/0 conn(0x7f6cac0d3670 legacy=0x7f6cac06c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.131 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.130+0000 7f6cb0c00640 1 -- 172.21.15.53:0/1648678133 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac0d2e70 con 0x7f6cac15dcc0 2024-03-08T06:03:44.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.131+0000 7f6cb0c00640 1 --2- 172.21.15.53:0/1648678133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6cac15dcc0 0x7f6cac0d1f80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009800 tx=0x7f6ca002b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee02a1de25d5e3f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:44.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.132+0000 7f6cab400640 1 -- 172.21.15.53:0/1648678133 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0034070 con 0x7f6cac15dcc0 2024-03-08T06:03:44.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.132+0000 7f6cab400640 1 -- 172.21.15.53:0/1648678133 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca0030880 con 0x7f6cac15dcc0 2024-03-08T06:03:44.132 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.132+0000 7f6cab400640 1 -- 172.21.15.53:0/1648678133 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0030c50 con 0x7f6cac15dcc0 2024-03-08T06:03:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.132+0000 7f6cb2000640 1 -- 172.21.15.53:0/1648678133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6cac15dcc0 msgr2=0x7f6cac0d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.132+0000 7f6cb2000640 1 --2- 172.21.15.53:0/1648678133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6cac15dcc0 0x7f6cac0d1f80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009800 tx=0x7f6ca002b9c0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.132+0000 7f6cb2000640 1 -- 172.21.15.53:0/1648678133 shutdown_connections 2024-03-08T06:03:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.132+0000 7f6cb2000640 1 --2- 172.21.15.53:0/1648678133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6cac15dcc0 0x7f6cac0d1f80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.132+0000 7f6cb2000640 1 -- 172.21.15.53:0/1648678133 >> 172.21.15.53:0/1648678133 conn(0x7f6cac066fc0 msgr2=0x7f6cac0d3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.132+0000 7f6cb2000640 1 -- 172.21.15.53:0/1648678133 shutdown_connections 2024-03-08T06:03:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.132+0000 7f6cb2000640 1 -- 172.21.15.53:0/1648678133 wait complete. 2024-03-08T06:03:44.134 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:44.133+0000 7f5845a42040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T06:03:44.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.134+0000 7f6cb2000640 1 Processor -- start 2024-03-08T06:03:44.135 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.134+0000 7f6cb2000640 1 -- start start 2024-03-08T06:03:44.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.135+0000 7f6cb2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cac06c9a0 0x7f6cac14d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.135+0000 7f6cb2000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cac0d3670 0x7f6cac151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.135+0000 7f6cb2000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6cac15dcc0 0x7f6cac0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.135+0000 7f6cb2000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac15e220 con 0x7f6cac0d3670 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.135+0000 7f6cb2000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac15e0a0 con 0x7f6cac06c9a0 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.135+0000 7f6cb2000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac0b9620 con 0x7f6cac15dcc0 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.135+0000 7f6cb0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cac06c9a0 0x7f6cac14d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.135+0000 7f6cb0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cac06c9a0 0x7f6cac14d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54224/0 (socket says 172.21.15.53:54224) 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.136+0000 7f6cb0c00640 1 -- 172.21.15.53:0/2810528556 learned_addr learned my addr 172.21.15.53:0/2810528556 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.136+0000 7f6cb1600640 1 --2- 172.21.15.53:0/2810528556 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6cac15dcc0 0x7f6cac0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.136+0000 7f6cabe00640 1 --2- 172.21.15.53:0/2810528556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cac0d3670 0x7f6cac151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.136+0000 7f6cb0c00640 1 -- 172.21.15.53:0/2810528556 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6cac15dcc0 msgr2=0x7f6cac0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.136+0000 7f6cb0c00640 1 --2- 172.21.15.53:0/2810528556 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6cac15dcc0 0x7f6cac0cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.136+0000 7f6cb0c00640 1 -- 172.21.15.53:0/2810528556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cac0d3670 msgr2=0x7f6cac151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.136+0000 7f6cb0c00640 1 --2- 172.21.15.53:0/2810528556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cac0d3670 0x7f6cac151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.136+0000 7f6cb0c00640 1 -- 172.21.15.53:0/2810528556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac0cae80 con 0x7f6cac06c9a0 2024-03-08T06:03:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.136+0000 7f6cabe00640 1 --2- 172.21.15.53:0/2810528556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cac0d3670 0x7f6cac151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:03:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.136+0000 7f6cb0c00640 1 --2- 172.21.15.53:0/2810528556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cac06c9a0 0x7f6cac14d450 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009920 tx=0x7f6ca00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.137+0000 7f6ca8c00640 1 -- 172.21.15.53:0/2810528556 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0034040 con 0x7f6cac06c9a0 2024-03-08T06:03:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.137+0000 7f6ca8c00640 1 -- 172.21.15.53:0/2810528556 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca00443f0 con 0x7f6cac06c9a0 2024-03-08T06:03:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.137+0000 7f6ca8c00640 1 -- 172.21.15.53:0/2810528556 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00447e0 con 0x7f6cac06c9a0 2024-03-08T06:03:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.137+0000 7f6cb2000640 1 -- 172.21.15.53:0/2810528556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac150990 con 0x7f6cac06c9a0 2024-03-08T06:03:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.137+0000 7f6cb2000640 1 -- 172.21.15.53:0/2810528556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac150c30 con 0x7f6cac06c9a0 2024-03-08T06:03:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.137+0000 7f6ca8c00640 1 -- 172.21.15.53:0/2810528556 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6ca004b060 con 0x7f6cac06c9a0 2024-03-08T06:03:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.137+0000 7f6ca8c00640 1 -- 172.21.15.53:0/2810528556 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f6ca0030740 con 0x7f6cac06c9a0 2024-03-08T06:03:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.138+0000 7f6cb2000640 1 -- 172.21.15.53:0/2810528556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac075100 con 0x7f6cac06c9a0 2024-03-08T06:03:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.142+0000 7f6ca8c00640 1 -- 172.21.15.53:0/2810528556 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6ca0063dd0 con 0x7f6cac06c9a0 2024-03-08T06:03:44.188 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:44.188+0000 7fdee322b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T06:03:44.197 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:44.197+0000 7f5845a42040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T06:03:44.224 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.224+0000 7f6cb2000640 1 -- 172.21.15.53:0/2810528556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9296f075-c6f9-416e-99dd-13e3db81e1be", "id": 2} v 0) v1 -- 0x7f6cac06c430 con 0x7f6cac06c9a0 2024-03-08T06:03:44.232 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.231+0000 7f6ca8c00640 1 -- 172.21.15.53:0/2810528556 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9296f075-c6f9-416e-99dd-13e3db81e1be", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6ca0052030 con 0x7f6cac06c9a0 2024-03-08T06:03:44.232 INFO:teuthology.orchestra.run.smithi053.stdout:2 2024-03-08T06:03:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.233+0000 7f6cb2000640 1 -- 172.21.15.53:0/2810528556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cac06c9a0 msgr2=0x7f6cac14d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.233+0000 7f6cb2000640 1 --2- 172.21.15.53:0/2810528556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cac06c9a0 0x7f6cac14d450 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009920 tx=0x7f6ca00046c0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.233+0000 7f6cb2000640 1 -- 172.21.15.53:0/2810528556 shutdown_connections 2024-03-08T06:03:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.233+0000 7f6cb2000640 1 --2- 172.21.15.53:0/2810528556 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6cac15dcc0 0x7f6cac0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.233+0000 7f6cb2000640 1 --2- 172.21.15.53:0/2810528556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cac0d3670 0x7f6cac151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.233+0000 7f6cb2000640 1 --2- 172.21.15.53:0/2810528556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cac06c9a0 0x7f6cac14d450 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.233+0000 7f6cb2000640 1 -- 172.21.15.53:0/2810528556 >> 172.21.15.53:0/2810528556 conn(0x7f6cac066fc0 msgr2=0x7f6cac15b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.233+0000 7f6cb2000640 1 -- 172.21.15.53:0/2810528556 shutdown_connections 2024-03-08T06:03:44.234 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.233+0000 7f6cb2000640 1 -- 172.21.15.53:0/2810528556 wait complete. 2024-03-08T06:03:44.249 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:44.249+0000 7fdee322b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T06:03:44.251 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 2d13fea2-46c6-4f71-9031-cdfe8b87de91 3 2024-03-08T06:03:44.255 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:44.255+0000 7f5845a42040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T06:03:44.315 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:44.314+0000 7fdee322b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T06:03:44.376 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:44.375+0000 7fdee322b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T06:03:44.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.476+0000 7f972a600640 1 Processor -- start 2024-03-08T06:03:44.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.476+0000 7f972a600640 1 -- start start 2024-03-08T06:03:44.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.477+0000 7f972a600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f97240b5e80 0x7f97240ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.477+0000 7f972a600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f97240d2b20 0x7f97240cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.477+0000 7f972a600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f972415e250 con 0x7f972415de70 2024-03-08T06:03:44.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.477+0000 7f972a600640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97240cefd0 con 0x7f97240b5e80 2024-03-08T06:03:44.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.477+0000 7f972a600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97240cf1f0 con 0x7f97240d2b20 2024-03-08T06:03:44.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.478+0000 7f971aa00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f97240b5e80 0x7f97240ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.478+0000 7f9723e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f97240d2b20 0x7f97240cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.478+0000 7f971aa00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f97240b5e80 0x7f97240ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54232/0 (socket says 172.21.15.53:54232) 2024-03-08T06:03:44.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.478+0000 7f971aa00640 1 -- 172.21.15.53:0/2746086841 learned_addr learned my addr 172.21.15.53:0/2746086841 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-08T06:03:44.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.478+0000 7f9723e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f97240d2b20 0x7f97240cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:53298/0 (socket says 172.21.15.53:53298) 2024-03-08T06:03:44.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.478+0000 7f9723e00640 1 -- 172.21.15.53:0/2746086841 >> v2:172.21.15.53:3300/0 conn(0x7f97240b5e80 msgr2=0x7f97240ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:44.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.478+0000 7f9723e00640 1 --2- 172.21.15.53:0/2746086841 >> v2:172.21.15.53:3300/0 conn(0x7f97240b5e80 0x7f97240ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.478+0000 7f9723e00640 1 -- 172.21.15.53:0/2746086841 >> v1:172.21.15.40:6789/0 conn(0x7f972415de70 legacy=0x7f97240b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.478+0000 7f9723e00640 1 -- 172.21.15.53:0/2746086841 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240d1d60 con 0x7f97240d2b20 2024-03-08T06:03:44.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.479+0000 7f9723e00640 1 --2- 172.21.15.53:0/2746086841 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f97240d2b20 0x7f97240cea90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f971c009d90 tx=0x7f971c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6e52d29045ec7d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:44.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.480+0000 7f9722a00640 1 -- 172.21.15.53:0/2746086841 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c039070 con 0x7f97240d2b20 2024-03-08T06:03:44.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.480+0000 7f9722a00640 1 -- 172.21.15.53:0/2746086841 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c02cc50 con 0x7f97240d2b20 2024-03-08T06:03:44.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.480+0000 7f9722a00640 1 -- 172.21.15.53:0/2746086841 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c034440 con 0x7f97240d2b20 2024-03-08T06:03:44.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.480+0000 7f972a600640 1 -- 172.21.15.53:0/2746086841 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f97240d2b20 msgr2=0x7f97240cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.480+0000 7f972a600640 1 --2- 172.21.15.53:0/2746086841 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f97240d2b20 0x7f97240cea90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f971c009d90 tx=0x7f971c02bea0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.480+0000 7f972a600640 1 -- 172.21.15.53:0/2746086841 shutdown_connections 2024-03-08T06:03:44.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.480+0000 7f972a600640 1 --2- 172.21.15.53:0/2746086841 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f97240d2b20 0x7f97240cea90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.480+0000 7f972a600640 1 --2- 172.21.15.53:0/2746086841 >> v2:172.21.15.53:3300/0 conn(0x7f97240b5e80 0x7f97240ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.480+0000 7f972a600640 1 -- 172.21.15.53:0/2746086841 >> 172.21.15.53:0/2746086841 conn(0x7f9724067240 msgr2=0x7f97240d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:44.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.480+0000 7f972a600640 1 -- 172.21.15.53:0/2746086841 shutdown_connections 2024-03-08T06:03:44.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.481+0000 7f972a600640 1 -- 172.21.15.53:0/2746086841 wait complete. 2024-03-08T06:03:44.482 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.482+0000 7f972a600640 1 Processor -- start 2024-03-08T06:03:44.483 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.482+0000 7f972a600640 1 -- start start 2024-03-08T06:03:44.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.483+0000 7f972a600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97240b5e80 0x7f97241531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.485 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.483+0000 7f972a600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97240d2b20 0x7f97240af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.483+0000 7f972a600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f972415de70 0x7f97240aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.483+0000 7f972a600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97240d2600 con 0x7f97240d2b20 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.483+0000 7f972a600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97240d2320 con 0x7f97240b5e80 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.483+0000 7f972a600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97240b6d80 con 0x7f972415de70 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.484+0000 7f9723400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97240b5e80 0x7f97241531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.484+0000 7f9723e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f972415de70 0x7f97240aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.484+0000 7f9723400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97240b5e80 0x7f97241531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54242/0 (socket says 172.21.15.53:54242) 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.484+0000 7f9723400640 1 -- 172.21.15.53:0/1628083246 learned_addr learned my addr 172.21.15.53:0/1628083246 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.484+0000 7f971aa00640 1 --2- 172.21.15.53:0/1628083246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97240d2b20 0x7f97240af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.484+0000 7f9723e00640 1 -- 172.21.15.53:0/1628083246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97240b5e80 msgr2=0x7f97241531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.484+0000 7f9723e00640 1 --2- 172.21.15.53:0/1628083246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97240b5e80 0x7f97241531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.484+0000 7f9723e00640 1 -- 172.21.15.53:0/1628083246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97240d2b20 msgr2=0x7f97240af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.484+0000 7f9723e00640 1 --2- 172.21.15.53:0/1628083246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97240d2b20 0x7f97240af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.484+0000 7f9723e00640 1 -- 172.21.15.53:0/1628083246 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240ab9e0 con 0x7f972415de70 2024-03-08T06:03:44.486 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.484+0000 7f9723e00640 1 --2- 172.21.15.53:0/1628083246 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f972415de70 0x7f97240aae70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f971c033000 tx=0x7f971c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:44.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.485+0000 7f971be00640 1 -- 172.21.15.53:0/1628083246 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c039050 con 0x7f972415de70 2024-03-08T06:03:44.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.485+0000 7f971be00640 1 -- 172.21.15.53:0/1628083246 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c045070 con 0x7f972415de70 2024-03-08T06:03:44.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.485+0000 7f971be00640 1 -- 172.21.15.53:0/1628083246 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c009450 con 0x7f972415de70 2024-03-08T06:03:44.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.485+0000 7f972a600640 1 -- 172.21.15.53:0/1628083246 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97240abcc0 con 0x7f972415de70 2024-03-08T06:03:44.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.485+0000 7f972a600640 1 -- 172.21.15.53:0/1628083246 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97240ac280 con 0x7f972415de70 2024-03-08T06:03:44.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.485+0000 7f971be00640 1 -- 172.21.15.53:0/1628083246 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f971c03e610 con 0x7f972415de70 2024-03-08T06:03:44.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.485+0000 7f971be00640 1 -- 172.21.15.53:0/1628083246 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f971c03ed90 con 0x7f972415de70 2024-03-08T06:03:44.487 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.486+0000 7f972a600640 1 -- 172.21.15.53:0/1628083246 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9724067cb0 con 0x7f972415de70 2024-03-08T06:03:44.489 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.489+0000 7f971be00640 1 -- 172.21.15.53:0/1628083246 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f971c03e900 con 0x7f972415de70 2024-03-08T06:03:44.568 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.568+0000 7f972a600640 1 -- 172.21.15.53:0/1628083246 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2d13fea2-46c6-4f71-9031-cdfe8b87de91", "id": 3} v 0) v1 -- 0x7f9724074870 con 0x7f972415de70 2024-03-08T06:03:44.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.576+0000 7f971be00640 1 -- 172.21.15.53:0/1628083246 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2d13fea2-46c6-4f71-9031-cdfe8b87de91", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f971c03eb60 con 0x7f972415de70 2024-03-08T06:03:44.577 INFO:teuthology.orchestra.run.smithi053.stdout:3 2024-03-08T06:03:44.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.577+0000 7f972a600640 1 -- 172.21.15.53:0/1628083246 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f972415de70 msgr2=0x7f97240aae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.577+0000 7f972a600640 1 --2- 172.21.15.53:0/1628083246 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f972415de70 0x7f97240aae70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f971c033000 tx=0x7f971c004a40 comp rx=0 tx=0).stop 2024-03-08T06:03:44.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.578+0000 7f972a600640 1 -- 172.21.15.53:0/1628083246 shutdown_connections 2024-03-08T06:03:44.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.578+0000 7f972a600640 1 --2- 172.21.15.53:0/1628083246 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f972415de70 0x7f97240aae70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.578+0000 7f972a600640 1 --2- 172.21.15.53:0/1628083246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97240d2b20 0x7f97240af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.578+0000 7f972a600640 1 --2- 172.21.15.53:0/1628083246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97240b5e80 0x7f97241531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.578+0000 7f972a600640 1 -- 172.21.15.53:0/1628083246 >> 172.21.15.53:0/1628083246 conn(0x7f9724067240 msgr2=0x7f97240b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:44.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.578+0000 7f972a600640 1 -- 172.21.15.53:0/1628083246 shutdown_connections 2024-03-08T06:03:44.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.578+0000 7f972a600640 1 -- 172.21.15.53:0/1628083246 wait complete. 2024-03-08T06:03:44.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 75452991-232c-4cf9-b699-5c9471173669 4 2024-03-08T06:03:44.729 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:44.729+0000 7f5845a42040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T06:03:44.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.834+0000 7f9fdd600640 1 Processor -- start 2024-03-08T06:03:44.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.834+0000 7f9fdd600640 1 -- start start 2024-03-08T06:03:44.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.835+0000 7f9fdd600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9fd80b1f50 0x7f9fd80ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.836+0000 7f9fdd600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd815c790 con 0x7f9fd815c020 2024-03-08T06:03:44.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.836+0000 7f9fdd600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd815ca70 con 0x7f9fd815fcd0 2024-03-08T06:03:44.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.836+0000 7f9fdd600640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd815cd50 con 0x7f9fd80b1f50 2024-03-08T06:03:44.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.836+0000 7f9fd7e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9fd80b1f50 0x7f9fd80ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.836+0000 7f9fdcc00640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9fd815fcd0 0x7f9fd806ac50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57962/0 (socket says 172.21.15.53:57962) 2024-03-08T06:03:44.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.836+0000 7f9fdcc00640 1 -- 172.21.15.53:0/1497103685 learned_addr learned my addr 172.21.15.53:0/1497103685 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-03-08T06:03:44.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.836+0000 7f9fd7e00640 1 -- 172.21.15.53:0/1497103685 >> v1:172.21.15.53:6789/0 conn(0x7f9fd815fcd0 legacy=0x7f9fd806ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.836+0000 7f9fd7e00640 1 -- 172.21.15.53:0/1497103685 >> v1:172.21.15.40:6789/0 conn(0x7f9fd815c020 legacy=0x7f9fd80cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:44.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.836+0000 7f9fd7e00640 1 -- 172.21.15.53:0/1497103685 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd815f7b0 con 0x7f9fd80b1f50 2024-03-08T06:03:44.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.837+0000 7f9fd7e00640 1 --2- 172.21.15.53:0/1497103685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd80b1f50 0x7f9fd80ccc60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80097a0 tx=0x7f9fc802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=820deb79b7878e91 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:44.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.838+0000 7f9fd7400640 1 -- 172.21.15.53:0/1497103685 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80315a0 con 0x7f9fd80b1f50 2024-03-08T06:03:44.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.838+0000 7f9fd7400640 1 -- 172.21.15.53:0/1497103685 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc8037070 con 0x7f9fd80b1f50 2024-03-08T06:03:44.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.838+0000 7f9fd7400640 1 -- 172.21.15.53:0/1497103685 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8031960 con 0x7f9fd80b1f50 2024-03-08T06:03:44.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.838+0000 7f9fdd600640 1 -- 172.21.15.53:0/1497103685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd80b1f50 msgr2=0x7f9fd80ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.838+0000 7f9fdd600640 1 --2- 172.21.15.53:0/1497103685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd80b1f50 0x7f9fd80ccc60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80097a0 tx=0x7f9fc802b840 comp rx=0 tx=0).stop 2024-03-08T06:03:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.839+0000 7f9fdd600640 1 -- 172.21.15.53:0/1497103685 shutdown_connections 2024-03-08T06:03:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.839+0000 7f9fdd600640 1 --2- 172.21.15.53:0/1497103685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd80b1f50 0x7f9fd80ccc60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.839+0000 7f9fdd600640 1 -- 172.21.15.53:0/1497103685 >> 172.21.15.53:0/1497103685 conn(0x7f9fd8099a70 msgr2=0x7f9fd8099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.839+0000 7f9fdd600640 1 -- 172.21.15.53:0/1497103685 shutdown_connections 2024-03-08T06:03:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.839+0000 7f9fdd600640 1 -- 172.21.15.53:0/1497103685 wait complete. 2024-03-08T06:03:44.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.840+0000 7f9fdd600640 1 Processor -- start 2024-03-08T06:03:44.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.840+0000 7f9fdd600640 1 -- start start 2024-03-08T06:03:44.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.841+0000 7f9fdd600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80b1f50 0x7f9fd80ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.841+0000 7f9fdd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd815c020 0x7f9fd81503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.841+0000 7f9fdd600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9fd815fcd0 0x7f9fd80aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:44.843 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.841+0000 7f9fdd600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80cdaf0 con 0x7f9fd80b1f50 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.841+0000 7f9fdd600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80cd970 con 0x7f9fd815c020 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.841+0000 7f9fdd600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd806b360 con 0x7f9fd815fcd0 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fcfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd815c020 0x7f9fd81503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fdcc00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9fd815fcd0 0x7f9fd80aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fdcc00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9fd815fcd0 0x7f9fd80aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:53310/0 (socket says 172.21.15.53:53310) 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fdcc00640 1 -- 172.21.15.53:0/668928129 learned_addr learned my addr 172.21.15.53:0/668928129 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fcfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd815c020 0x7f9fd81503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54260/0 (socket says 172.21.15.53:54260) 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fd7e00640 1 --2- 172.21.15.53:0/668928129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80b1f50 0x7f9fd80ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fdcc00640 1 -- 172.21.15.53:0/668928129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd815c020 msgr2=0x7f9fd81503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fdcc00640 1 --2- 172.21.15.53:0/668928129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd815c020 0x7f9fd81503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fdcc00640 1 -- 172.21.15.53:0/668928129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80b1f50 msgr2=0x7f9fd80ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fdcc00640 1 --2- 172.21.15.53:0/668928129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80b1f50 0x7f9fd80ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fdcc00640 1 -- 172.21.15.53:0/668928129 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd80c9390 con 0x7f9fd815fcd0 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fcfe00640 1 --2- 172.21.15.53:0/668928129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd815c020 0x7f9fd81503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:03:44.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.842+0000 7f9fdcc00640 1 --2- 172.21.15.53:0/668928129 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9fd815fcd0 0x7f9fd80aea20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000cab0 tx=0x7f9fd000cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:44.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.843+0000 7f9fd4c00640 1 -- 172.21.15.53:0/668928129 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0017070 con 0x7f9fd815fcd0 2024-03-08T06:03:44.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.843+0000 7f9fd4c00640 1 -- 172.21.15.53:0/668928129 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd000ac30 con 0x7f9fd815fcd0 2024-03-08T06:03:44.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.843+0000 7f9fd4c00640 1 -- 172.21.15.53:0/668928129 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0012400 con 0x7f9fd815fcd0 2024-03-08T06:03:44.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.843+0000 7f9fdd600640 1 -- 172.21.15.53:0/668928129 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd80c9670 con 0x7f9fd815fcd0 2024-03-08T06:03:44.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.843+0000 7f9fdd600640 1 -- 172.21.15.53:0/668928129 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd80c9c30 con 0x7f9fd815fcd0 2024-03-08T06:03:44.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.843+0000 7f9fd4c00640 1 -- 172.21.15.53:0/668928129 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9fd001a070 con 0x7f9fd815fcd0 2024-03-08T06:03:44.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.843+0000 7f9fd4c00640 1 -- 172.21.15.53:0/668928129 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9fd001cdd0 con 0x7f9fd815fcd0 2024-03-08T06:03:44.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.844+0000 7f9fdd600640 1 -- 172.21.15.53:0/668928129 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd8075100 con 0x7f9fd815fcd0 2024-03-08T06:03:44.847 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.846+0000 7f9fd4c00640 1 -- 172.21.15.53:0/668928129 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9fd001c880 con 0x7f9fd815fcd0 2024-03-08T06:03:44.868 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:44.868+0000 7fdee322b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T06:03:44.899 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:44.899+0000 7f5845a42040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T06:03:44.928 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.927+0000 7f9fdd600640 1 -- 172.21.15.53:0/668928129 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "75452991-232c-4cf9-b699-5c9471173669", "id": 4} v 0) v1 -- 0x7f9fd8159de0 con 0x7f9fd815fcd0 2024-03-08T06:03:44.936 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.935+0000 7f9fd4c00640 1 -- 172.21.15.53:0/668928129 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "75452991-232c-4cf9-b699-5c9471173669", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9fd001ca60 con 0x7f9fd815fcd0 2024-03-08T06:03:44.936 INFO:teuthology.orchestra.run.smithi053.stdout:4 2024-03-08T06:03:44.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.937+0000 7f9fdd600640 1 -- 172.21.15.53:0/668928129 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9fd815fcd0 msgr2=0x7f9fd80aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:44.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.937+0000 7f9fdd600640 1 --2- 172.21.15.53:0/668928129 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9fd815fcd0 0x7f9fd80aea20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000cab0 tx=0x7f9fd000cf70 comp rx=0 tx=0).stop 2024-03-08T06:03:44.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.937+0000 7f9fdd600640 1 -- 172.21.15.53:0/668928129 shutdown_connections 2024-03-08T06:03:44.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.937+0000 7f9fdd600640 1 --2- 172.21.15.53:0/668928129 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9fd815fcd0 0x7f9fd80aea20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.937+0000 7f9fdd600640 1 --2- 172.21.15.53:0/668928129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd815c020 0x7f9fd81503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.937+0000 7f9fdd600640 1 --2- 172.21.15.53:0/668928129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd80b1f50 0x7f9fd80ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:44.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.937+0000 7f9fdd600640 1 -- 172.21.15.53:0/668928129 >> 172.21.15.53:0/668928129 conn(0x7f9fd8099a70 msgr2=0x7f9fd806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:44.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.937+0000 7f9fdd600640 1 -- 172.21.15.53:0/668928129 shutdown_connections 2024-03-08T06:03:44.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:44.937+0000 7f9fdd600640 1 -- 172.21.15.53:0/668928129 wait complete. 2024-03-08T06:03:44.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new c1fdd926-f679-471c-b49a-98bbe23ab014 5 2024-03-08T06:03:45.044 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:45.043+0000 7fdee322b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T06:03:45.051 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:45.051+0000 7f5845a42040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T06:03:45.107 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:45.107+0000 7f5845a42040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T06:03:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.139+0000 7fae5ea00640 1 Processor -- start 2024-03-08T06:03:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.139+0000 7fae5ea00640 1 -- start start 2024-03-08T06:03:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.141+0000 7fae5ea00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae580d1df0 0x7fae580cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.141+0000 7fae5ea00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae580b6d80 con 0x7fae580b5e80 2024-03-08T06:03:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.141+0000 7fae5ea00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae580d2460 con 0x7fae5815df90 2024-03-08T06:03:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.141+0000 7fae5ea00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae580d3660 con 0x7fae580d1df0 2024-03-08T06:03:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.141+0000 7fae5ca00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae580d1df0 0x7fae580cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.141+0000 7fae57e00640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fae5815df90 0x7fae580b5770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57964/0 (socket says 172.21.15.53:57964) 2024-03-08T06:03:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.141+0000 7fae57e00640 1 -- 172.21.15.53:0/2835939581 learned_addr learned my addr 172.21.15.53:0/2835939581 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-03-08T06:03:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.142+0000 7fae5ca00640 1 -- 172.21.15.53:0/2835939581 >> v1:172.21.15.53:6789/0 conn(0x7fae5815df90 legacy=0x7fae580b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.142+0000 7fae5ca00640 1 -- 172.21.15.53:0/2835939581 >> v1:172.21.15.40:6789/0 conn(0x7fae580b5e80 legacy=0x7fae580b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.142+0000 7fae5ca00640 1 -- 172.21.15.53:0/2835939581 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580d2740 con 0x7fae580d1df0 2024-03-08T06:03:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.143+0000 7fae5ca00640 1 --2- 172.21.15.53:0/2835939581 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae580d1df0 0x7fae580cef60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae50009dc0 tx=0x7fae5002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd83b4732d6d1a11 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.143+0000 7fae57400640 1 -- 172.21.15.53:0/2835939581 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50039070 con 0x7fae580d1df0 2024-03-08T06:03:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.143+0000 7fae57400640 1 -- 172.21.15.53:0/2835939581 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5002ccd0 con 0x7fae580d1df0 2024-03-08T06:03:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.143+0000 7fae57400640 1 -- 172.21.15.53:0/2835939581 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50034660 con 0x7fae580d1df0 2024-03-08T06:03:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.143+0000 7fae5ea00640 1 -- 172.21.15.53:0/2835939581 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae580d1df0 msgr2=0x7fae580cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.143+0000 7fae5ea00640 1 --2- 172.21.15.53:0/2835939581 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae580d1df0 0x7fae580cef60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae50009dc0 tx=0x7fae5002bf20 comp rx=0 tx=0).stop 2024-03-08T06:03:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.144+0000 7fae5ea00640 1 -- 172.21.15.53:0/2835939581 shutdown_connections 2024-03-08T06:03:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.144+0000 7fae5ea00640 1 --2- 172.21.15.53:0/2835939581 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae580d1df0 0x7fae580cef60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.144+0000 7fae5ea00640 1 -- 172.21.15.53:0/2835939581 >> 172.21.15.53:0/2835939581 conn(0x7fae58067240 msgr2=0x7fae580d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.144+0000 7fae5ea00640 1 -- 172.21.15.53:0/2835939581 shutdown_connections 2024-03-08T06:03:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.144+0000 7fae5ea00640 1 -- 172.21.15.53:0/2835939581 wait complete. 2024-03-08T06:03:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.145+0000 7fae5ea00640 1 Processor -- start 2024-03-08T06:03:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.145+0000 7fae5ea00640 1 -- start start 2024-03-08T06:03:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.146+0000 7fae5ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae580b5e80 0x7fae58152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.146+0000 7fae5ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae580d1df0 0x7fae5815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.146+0000 7fae5ea00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae5815df90 0x7fae580c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.146+0000 7fae5ea00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5815e370 con 0x7fae580b5e80 2024-03-08T06:03:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.146+0000 7fae5ea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580d2d00 con 0x7fae580d1df0 2024-03-08T06:03:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.146+0000 7fae5ea00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae580b7970 con 0x7fae5815df90 2024-03-08T06:03:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae5ca00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae5815df90 0x7fae580c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae5ca00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae5815df90 0x7fae580c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:53332/0 (socket says 172.21.15.53:53332) 2024-03-08T06:03:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae5ca00640 1 -- 172.21.15.53:0/3302647908 learned_addr learned my addr 172.21.15.53:0/3302647908 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae4fe00640 1 --2- 172.21.15.53:0/3302647908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae580d1df0 0x7fae5815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae57e00640 1 --2- 172.21.15.53:0/3302647908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae580b5e80 0x7fae58152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae5ca00640 1 -- 172.21.15.53:0/3302647908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae580d1df0 msgr2=0x7fae5815ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae5ca00640 1 --2- 172.21.15.53:0/3302647908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae580d1df0 0x7fae5815ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae5ca00640 1 -- 172.21.15.53:0/3302647908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae580b5e80 msgr2=0x7fae58152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae5ca00640 1 --2- 172.21.15.53:0/3302647908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae580b5e80 0x7fae58152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae5ca00640 1 -- 172.21.15.53:0/3302647908 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58152940 con 0x7fae5815df90 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae4fe00640 1 --2- 172.21.15.53:0/3302647908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae580d1df0 0x7fae5815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.147+0000 7fae5ca00640 1 --2- 172.21.15.53:0/3302647908 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae5815df90 0x7fae580c8e50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fae50037040 tx=0x7fae50004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.148+0000 7fae54c00640 1 -- 172.21.15.53:0/3302647908 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50039050 con 0x7fae5815df90 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.148+0000 7fae54c00640 1 -- 172.21.15.53:0/3302647908 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae50045070 con 0x7fae5815df90 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.148+0000 7fae5ea00640 1 -- 172.21.15.53:0/3302647908 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae580b2ca0 con 0x7fae5815df90 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.148+0000 7fae54c00640 1 -- 172.21.15.53:0/3302647908 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5003e850 con 0x7fae5815df90 2024-03-08T06:03:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.148+0000 7fae5ea00640 1 -- 172.21.15.53:0/3302647908 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae580b3260 con 0x7fae5815df90 2024-03-08T06:03:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.148+0000 7fae54c00640 1 -- 172.21.15.53:0/3302647908 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fae5003e9f0 con 0x7fae5815df90 2024-03-08T06:03:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.148+0000 7fae54c00640 1 -- 172.21.15.53:0/3302647908 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fae5003ece0 con 0x7fae5815df90 2024-03-08T06:03:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.149+0000 7fae5ea00640 1 -- 172.21.15.53:0/3302647908 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae58152b40 con 0x7fae5815df90 2024-03-08T06:03:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.152+0000 7fae54c00640 1 -- 172.21.15.53:0/3302647908 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fae58152b40 con 0x7fae5815df90 2024-03-08T06:03:45.180 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:45.180+0000 7fdee322b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T06:03:45.231 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.231+0000 7fae5ea00640 1 -- 172.21.15.53:0/3302647908 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c1fdd926-f679-471c-b49a-98bbe23ab014", "id": 5} v 0) v1 -- 0x7fae58067cb0 con 0x7fae5815df90 2024-03-08T06:03:45.238 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.237+0000 7fae54c00640 1 -- 172.21.15.53:0/3302647908 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c1fdd926-f679-471c-b49a-98bbe23ab014", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fae5004e020 con 0x7fae5815df90 2024-03-08T06:03:45.238 INFO:teuthology.orchestra.run.smithi053.stdout:5 2024-03-08T06:03:45.238 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:45.238+0000 7fdee322b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T06:03:45.239 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.238+0000 7fae5ea00640 1 -- 172.21.15.53:0/3302647908 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae5815df90 msgr2=0x7fae580c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.238+0000 7fae5ea00640 1 --2- 172.21.15.53:0/3302647908 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae5815df90 0x7fae580c8e50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fae50037040 tx=0x7fae50004a40 comp rx=0 tx=0).stop 2024-03-08T06:03:45.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.239+0000 7fae5ea00640 1 -- 172.21.15.53:0/3302647908 shutdown_connections 2024-03-08T06:03:45.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.239+0000 7fae5ea00640 1 --2- 172.21.15.53:0/3302647908 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fae5815df90 0x7fae580c8e50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.239+0000 7fae5ea00640 1 --2- 172.21.15.53:0/3302647908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae580d1df0 0x7fae5815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.239+0000 7fae5ea00640 1 --2- 172.21.15.53:0/3302647908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae580b5e80 0x7fae58152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.239+0000 7fae5ea00640 1 -- 172.21.15.53:0/3302647908 >> 172.21.15.53:0/3302647908 conn(0x7fae58067240 msgr2=0x7fae580c7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:45.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.239+0000 7fae5ea00640 1 -- 172.21.15.53:0/3302647908 shutdown_connections 2024-03-08T06:03:45.240 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.239+0000 7fae5ea00640 1 -- 172.21.15.53:0/3302647908 wait complete. 2024-03-08T06:03:45.240 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:45.241+0000 7f5845a42040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T06:03:45.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 0104e9cc-be81-45e7-ac1f-e0337a67a53c 6 2024-03-08T06:03:45.358 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:45.358+0000 7f5845a42040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T06:03:45.378 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:45.377+0000 7fdee322b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T06:03:45.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.441+0000 7f98e0000640 1 Processor -- start 2024-03-08T06:03:45.442 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.441+0000 7f98e0000640 1 -- start start 2024-03-08T06:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.442+0000 7f98e0000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98d8099e20 0x7f98d809a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.443+0000 7f98e0000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d80d01a0 con 0x7f98d80ccd20 2024-03-08T06:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.443+0000 7f98e0000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d80d0480 con 0x7f98d815bf90 2024-03-08T06:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.443+0000 7f98e0000640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d80d0760 con 0x7f98d8099e20 2024-03-08T06:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.443+0000 7f98dd600640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f98d815bf90 0x7f98d80cf750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57970/0 (socket says 172.21.15.53:57970) 2024-03-08T06:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.443+0000 7f98dd600640 1 -- 172.21.15.53:0/4184605059 learned_addr learned my addr 172.21.15.53:0/4184605059 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-03-08T06:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.443+0000 7f98dcc00640 1 --2- 172.21.15.53:0/4184605059 >> v2:172.21.15.40:3300/0 conn(0x7f98d8099e20 0x7f98d809a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.443+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475571889 0 0) 0x7f98d80d0480 con 0x7f98d815bf90 2024-03-08T06:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.443+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c0003520 con 0x7f98d815bf90 2024-03-08T06:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 <== mon.1 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 779348053 0 0) 0x7f98c0003520 con 0x7f98d815bf90 2024-03-08T06:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 >> v2:172.21.15.40:3300/0 conn(0x7f98d8099e20 msgr2=0x7f98d809a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 --2- 172.21.15.53:0/4184605059 >> v2:172.21.15.40:3300/0 conn(0x7f98d8099e20 0x7f98d809a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 >> v1:172.21.15.40:6789/0 conn(0x7f98d80ccd20 legacy=0x7f98d80cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80d0a40 con 0x7f98d815bf90 2024-03-08T06:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 <== mon.1 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f98cc001670 con 0x7f98d815bf90 2024-03-08T06:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 >> v1:172.21.15.53:6789/0 conn(0x7f98d815bf90 legacy=0x7f98d80cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 --2- 172.21.15.53:0/4184605059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c00039b0 0x7f98c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 --2- 172.21.15.53:0/4184605059 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98c00043b0 0x7f98c0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 --2- 172.21.15.53:0/4184605059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c0005080 0x7f98c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80d0760 con 0x7f98c00039b0 2024-03-08T06:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c000bfc0 con 0x7f98c0005080 2024-03-08T06:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c000c1c0 con 0x7f98c00043b0 2024-03-08T06:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98de000640 1 --2- 172.21.15.53:0/4184605059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c0005080 0x7f98c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.444+0000 7f98dcc00640 1 --2- 172.21.15.53:0/4184605059 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98c00043b0 0x7f98c0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.445+0000 7f98dd600640 1 --2- 172.21.15.53:0/4184605059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c00039b0 0x7f98c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.445+0000 7f98de000640 1 -- 172.21.15.53:0/4184605059 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98c00043b0 msgr2=0x7f98c0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.445+0000 7f98de000640 1 --2- 172.21.15.53:0/4184605059 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98c00043b0 0x7f98c0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.445+0000 7f98de000640 1 -- 172.21.15.53:0/4184605059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c00039b0 msgr2=0x7f98c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.445+0000 7f98de000640 1 --2- 172.21.15.53:0/4184605059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c00039b0 0x7f98c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.445+0000 7f98de000640 1 -- 172.21.15.53:0/4184605059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0003520 con 0x7f98c0005080 2024-03-08T06:03:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.446+0000 7f98de000640 1 -- 172.21.15.53:0/4184605059 reap_dead start 2024-03-08T06:03:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.446+0000 7f98de000640 1 --2- 172.21.15.53:0/4184605059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c0005080 0x7f98c000b9c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98d0001e50 tx=0x7f98c0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c56ab5feabfd320e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.446+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0001a70 con 0x7f98c0005080 2024-03-08T06:03:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.446+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d0003bb0 con 0x7f98c0005080 2024-03-08T06:03:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.446+0000 7f98d7e00640 1 -- 172.21.15.53:0/4184605059 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d00050d0 con 0x7f98c0005080 2024-03-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.447+0000 7f98e0000640 1 -- 172.21.15.53:0/4184605059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c0005080 msgr2=0x7f98c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.447+0000 7f98e0000640 1 --2- 172.21.15.53:0/4184605059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c0005080 0x7f98c000b9c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98d0001e50 tx=0x7f98c0005830 comp rx=0 tx=0).stop 2024-03-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.447+0000 7f98e0000640 1 -- 172.21.15.53:0/4184605059 shutdown_connections 2024-03-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.447+0000 7f98e0000640 1 --2- 172.21.15.53:0/4184605059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c0005080 0x7f98c000b9c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.447+0000 7f98e0000640 1 -- 172.21.15.53:0/4184605059 >> 172.21.15.53:0/4184605059 conn(0x7f98d80b3670 msgr2=0x7f98d80b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.447+0000 7f98e0000640 1 -- 172.21.15.53:0/4184605059 shutdown_connections 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.447+0000 7f98e0000640 1 -- 172.21.15.53:0/4184605059 wait complete. 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.449+0000 7f98e0000640 1 Processor -- start 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.449+0000 7f98e0000640 1 -- start start 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.450+0000 7f98e0000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98c0005080 0x7f98d81f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.450+0000 7f98e0000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d80b4730 0x7f98d80c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.450+0000 7f98e0000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98d80b7120 0x7f98d80caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.450+0000 7f98e0000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d815c910 con 0x7f98d80b4730 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.450+0000 7f98e0000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d815c790 con 0x7f98d80b7120 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.450+0000 7f98e0000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80cc700 con 0x7f98c0005080 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.450+0000 7f98de000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98d80b7120 0x7f98d80caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.450+0000 7f98de000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98d80b7120 0x7f98d80caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54280/0 (socket says 172.21.15.53:54280) 2024-03-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.450+0000 7f98de000640 1 -- 172.21.15.53:0/3908030373 learned_addr learned my addr 172.21.15.53:0/3908030373 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.450+0000 7f98dcc00640 1 --2- 172.21.15.53:0/3908030373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d80b4730 0x7f98d80c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.450+0000 7f98dd600640 1 --2- 172.21.15.53:0/3908030373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98c0005080 0x7f98d81f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.451+0000 7f98dd600640 1 -- 172.21.15.53:0/3908030373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98d80b7120 msgr2=0x7f98d80caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.451+0000 7f98dd600640 1 --2- 172.21.15.53:0/3908030373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98d80b7120 0x7f98d80caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.451+0000 7f98dd600640 1 -- 172.21.15.53:0/3908030373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d80b4730 msgr2=0x7f98d80c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.451+0000 7f98dd600640 1 --2- 172.21.15.53:0/3908030373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d80b4730 0x7f98d80c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.451+0000 7f98dd600640 1 -- 172.21.15.53:0/3908030373 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80c7b30 con 0x7f98c0005080 2024-03-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.451+0000 7f98dcc00640 1 --2- 172.21.15.53:0/3908030373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d80b4730 0x7f98d80c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.451+0000 7f98dd600640 1 --2- 172.21.15.53:0/3908030373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98c0005080 0x7f98d81f3930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f98cc001850 tx=0x7f98cc0084a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.451+0000 7f98d5600640 1 -- 172.21.15.53:0/3908030373 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc00f6e0 con 0x7f98c0005080 2024-03-08T06:03:45.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.452+0000 7f98d5600640 1 -- 172.21.15.53:0/3908030373 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc00f880 con 0x7f98c0005080 2024-03-08T06:03:45.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.452+0000 7f98d5600640 1 -- 172.21.15.53:0/3908030373 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc00fc30 con 0x7f98c0005080 2024-03-08T06:03:45.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.452+0000 7f98e0000640 1 -- 172.21.15.53:0/3908030373 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d80b4c10 con 0x7f98c0005080 2024-03-08T06:03:45.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.452+0000 7f98e0000640 1 -- 172.21.15.53:0/3908030373 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d80b51d0 con 0x7f98c0005080 2024-03-08T06:03:45.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.452+0000 7f98d5600640 1 -- 172.21.15.53:0/3908030373 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f98cc01f780 con 0x7f98c0005080 2024-03-08T06:03:45.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.452+0000 7f98d5600640 1 -- 172.21.15.53:0/3908030373 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f98cc00fdd0 con 0x7f98c0005080 2024-03-08T06:03:45.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.453+0000 7f98e0000640 1 -- 172.21.15.53:0/3908030373 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d80683c0 con 0x7f98c0005080 2024-03-08T06:03:45.457 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.455+0000 7f98d5600640 1 -- 172.21.15.53:0/3908030373 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f98cc015440 con 0x7f98c0005080 2024-03-08T06:03:45.500 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:45.500+0000 7fdee322b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T06:03:45.536 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.536+0000 7f98e0000640 1 -- 172.21.15.53:0/3908030373 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0104e9cc-be81-45e7-ac1f-e0337a67a53c", "id": 6} v 0) v1 -- 0x7f98d809b300 con 0x7f98c0005080 2024-03-08T06:03:45.544 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:45.543+0000 7f5845a42040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T06:03:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.545+0000 7f98d5600640 1 -- 172.21.15.53:0/3908030373 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0104e9cc-be81-45e7-ac1f-e0337a67a53c", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f98cc015680 con 0x7f98c0005080 2024-03-08T06:03:45.546 INFO:teuthology.orchestra.run.smithi053.stdout:6 2024-03-08T06:03:45.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.546+0000 7f98e0000640 1 -- 172.21.15.53:0/3908030373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98c0005080 msgr2=0x7f98d81f3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.547 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.546+0000 7f98e0000640 1 --2- 172.21.15.53:0/3908030373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98c0005080 0x7f98d81f3930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f98cc001850 tx=0x7f98cc0084a0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.546+0000 7f98e0000640 1 -- 172.21.15.53:0/3908030373 shutdown_connections 2024-03-08T06:03:45.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.546+0000 7f98e0000640 1 --2- 172.21.15.53:0/3908030373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98d80b7120 0x7f98d80caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.546+0000 7f98e0000640 1 --2- 172.21.15.53:0/3908030373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d80b4730 0x7f98d80c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.546+0000 7f98e0000640 1 --2- 172.21.15.53:0/3908030373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98c0005080 0x7f98d81f3930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.546+0000 7f98e0000640 1 -- 172.21.15.53:0/3908030373 >> 172.21.15.53:0/3908030373 conn(0x7f98d80b3670 msgr2=0x7f98d815d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:45.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.547+0000 7f98e0000640 1 -- 172.21.15.53:0/3908030373 shutdown_connections 2024-03-08T06:03:45.548 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.547+0000 7f98e0000640 1 -- 172.21.15.53:0/3908030373 wait complete. 2024-03-08T06:03:45.564 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 884629a7-a723-45c4-890c-b14a34610d7c 7 2024-03-08T06:03:45.596 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:45.596+0000 7f5845a42040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T06:03:45.693 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:45.692+0000 7fdee322b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T06:03:45.747 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:45.746+0000 7fdee322b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T06:03:45.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.761+0000 7fbb53e00640 1 Processor -- start 2024-03-08T06:03:45.762 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.761+0000 7fbb53e00640 1 -- start start 2024-03-08T06:03:45.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.763+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb540d1210 0x7fbb540cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.764 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.763+0000 7fbb53e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb540d0170 con 0x7fbb540b2e50 2024-03-08T06:03:45.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.763+0000 7fbb53e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb540d0450 con 0x7fbb54099c30 2024-03-08T06:03:45.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.763+0000 7fbb53e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540d0730 con 0x7fbb540d1210 2024-03-08T06:03:45.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.763+0000 7fbb52a00640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbb54099c30 0x7fbb5409a010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57974/0 (socket says 172.21.15.53:57974) 2024-03-08T06:03:45.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.763+0000 7fbb52a00640 1 -- 172.21.15.53:0/153199829 learned_addr learned my addr 172.21.15.53:0/153199829 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-03-08T06:03:45.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.763+0000 7fbb53400640 1 --2- 172.21.15.53:0/153199829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb540d1210 0x7fbb540cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.764+0000 7fbb52000640 1 -- 172.21.15.53:0/153199829 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514546818 0 0) 0x7fbb540d0450 con 0x7fbb54099c30 2024-03-08T06:03:45.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.764+0000 7fbb52000640 1 -- 172.21.15.53:0/153199829 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb38003520 con 0x7fbb54099c30 2024-03-08T06:03:45.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.764+0000 7fbb53400640 1 -- 172.21.15.53:0/153199829 >> v1:172.21.15.53:6789/0 conn(0x7fbb54099c30 legacy=0x7fbb5409a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.764+0000 7fbb53400640 1 -- 172.21.15.53:0/153199829 >> v1:172.21.15.40:6789/0 conn(0x7fbb540b2e50 legacy=0x7fbb540b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.765 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.764+0000 7fbb53400640 1 -- 172.21.15.53:0/153199829 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540d0a10 con 0x7fbb540d1210 2024-03-08T06:03:45.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.766+0000 7fbb53400640 1 --2- 172.21.15.53:0/153199829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb540d1210 0x7fbb540cfb00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009dc0 tx=0x7fbb4c02be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc93f300d83ddc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:45.766 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.766+0000 7fbb52000640 1 -- 172.21.15.53:0/153199829 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c039070 con 0x7fbb540d1210 2024-03-08T06:03:45.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.766+0000 7fbb52000640 1 -- 172.21.15.53:0/153199829 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c02cc30 con 0x7fbb540d1210 2024-03-08T06:03:45.767 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.766+0000 7fbb52000640 1 -- 172.21.15.53:0/153199829 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c034440 con 0x7fbb540d1210 2024-03-08T06:03:45.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.767+0000 7fbb53e00640 1 -- 172.21.15.53:0/153199829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb540d1210 msgr2=0x7fbb540cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.767+0000 7fbb53e00640 1 --2- 172.21.15.53:0/153199829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb540d1210 0x7fbb540cfb00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009dc0 tx=0x7fbb4c02be80 comp rx=0 tx=0).stop 2024-03-08T06:03:45.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.767+0000 7fbb53e00640 1 -- 172.21.15.53:0/153199829 shutdown_connections 2024-03-08T06:03:45.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.767+0000 7fbb53e00640 1 --2- 172.21.15.53:0/153199829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb540d1210 0x7fbb540cfb00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.767+0000 7fbb53e00640 1 -- 172.21.15.53:0/153199829 >> 172.21.15.53:0/153199829 conn(0x7fbb540b3460 msgr2=0x7fbb540b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:45.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.767+0000 7fbb53e00640 1 -- 172.21.15.53:0/153199829 shutdown_connections 2024-03-08T06:03:45.768 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.767+0000 7fbb53e00640 1 -- 172.21.15.53:0/153199829 wait complete. 2024-03-08T06:03:45.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.769+0000 7fbb53e00640 1 Processor -- start 2024-03-08T06:03:45.770 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.770+0000 7fbb53e00640 1 -- start start 2024-03-08T06:03:45.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.771+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb54099c30 0x7fbb5414dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.771+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb540b2e50 0x7fbb540af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.771+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb540d1210 0x7fbb54159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.771+0000 7fbb53e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5406cdd0 con 0x7fbb540d1210 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.771+0000 7fbb53e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5406cc50 con 0x7fbb540b2e50 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.771+0000 7fbb53e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5406cf50 con 0x7fbb54099c30 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.771+0000 7fbb52a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb54099c30 0x7fbb5414dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.771+0000 7fbb52a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb54099c30 0x7fbb5414dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:53364/0 (socket says 172.21.15.53:53364) 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.771+0000 7fbb52a00640 1 -- 172.21.15.53:0/3378624189 learned_addr learned my addr 172.21.15.53:0/3378624189 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.771+0000 7fbb4be00640 1 --2- 172.21.15.53:0/3378624189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb540b2e50 0x7fbb540af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb53400640 1 --2- 172.21.15.53:0/3378624189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb540d1210 0x7fbb54159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb52a00640 1 -- 172.21.15.53:0/3378624189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb540b2e50 msgr2=0x7fbb540af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb52a00640 1 --2- 172.21.15.53:0/3378624189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb540b2e50 0x7fbb540af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb52a00640 1 -- 172.21.15.53:0/3378624189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb540d1210 msgr2=0x7fbb54159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb52a00640 1 --2- 172.21.15.53:0/3378624189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb540d1210 0x7fbb54159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb52a00640 1 -- 172.21.15.53:0/3378624189 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540ae890 con 0x7fbb54099c30 2024-03-08T06:03:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb53400640 1 --2- 172.21.15.53:0/3378624189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb540d1210 0x7fbb54159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:03:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb52a00640 1 --2- 172.21.15.53:0/3378624189 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb54099c30 0x7fbb5414dd40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400c500 tx=0x7fbb4400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb4aa00640 1 -- 172.21.15.53:0/3378624189 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44017070 con 0x7fbb54099c30 2024-03-08T06:03:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb4aa00640 1 -- 172.21.15.53:0/3378624189 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb44005ce0 con 0x7fbb54099c30 2024-03-08T06:03:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb4aa00640 1 -- 172.21.15.53:0/3378624189 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44007330 con 0x7fbb54099c30 2024-03-08T06:03:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.772+0000 7fbb53e00640 1 -- 172.21.15.53:0/3378624189 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb540aeb70 con 0x7fbb54099c30 2024-03-08T06:03:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.773+0000 7fbb53e00640 1 -- 172.21.15.53:0/3378624189 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb540af130 con 0x7fbb54099c30 2024-03-08T06:03:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.773+0000 7fbb4aa00640 1 -- 172.21.15.53:0/3378624189 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbb4401a070 con 0x7fbb54099c30 2024-03-08T06:03:45.775 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.773+0000 7fbb4aa00640 1 -- 172.21.15.53:0/3378624189 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fbb440074d0 con 0x7fbb54099c30 2024-03-08T06:03:45.776 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.774+0000 7fbb53e00640 1 -- 172.21.15.53:0/3378624189 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb54075100 con 0x7fbb54099c30 2024-03-08T06:03:45.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.778+0000 7fbb4aa00640 1 -- 172.21.15.53:0/3378624189 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbb44021020 con 0x7fbb54099c30 2024-03-08T06:03:45.779 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:45.779+0000 7f5845a42040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T06:03:45.857 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.857+0000 7fbb53e00640 1 -- 172.21.15.53:0/3378624189 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "884629a7-a723-45c4-890c-b14a34610d7c", "id": 7} v 0) v1 -- 0x7fbb5415e5c0 con 0x7fbb54099c30 2024-03-08T06:03:45.865 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.864+0000 7fbb4aa00640 1 -- 172.21.15.53:0/3378624189 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "884629a7-a723-45c4-890c-b14a34610d7c", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbb4401c440 con 0x7fbb54099c30 2024-03-08T06:03:45.865 INFO:teuthology.orchestra.run.smithi053.stdout:7 2024-03-08T06:03:45.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.866+0000 7fbb53e00640 1 -- 172.21.15.53:0/3378624189 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb54099c30 msgr2=0x7fbb5414dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:45.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.866+0000 7fbb53e00640 1 --2- 172.21.15.53:0/3378624189 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb54099c30 0x7fbb5414dd40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400c500 tx=0x7fbb4400c9c0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.866+0000 7fbb53e00640 1 -- 172.21.15.53:0/3378624189 shutdown_connections 2024-03-08T06:03:45.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.866+0000 7fbb53e00640 1 --2- 172.21.15.53:0/3378624189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb540d1210 0x7fbb54159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.866+0000 7fbb53e00640 1 --2- 172.21.15.53:0/3378624189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb540b2e50 0x7fbb540af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.866+0000 7fbb53e00640 1 --2- 172.21.15.53:0/3378624189 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb54099c30 0x7fbb5414dd40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:45.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.866+0000 7fbb53e00640 1 -- 172.21.15.53:0/3378624189 >> 172.21.15.53:0/3378624189 conn(0x7fbb540b3460 msgr2=0x7fbb540cc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:45.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.866+0000 7fbb53e00640 1 -- 172.21.15.53:0/3378624189 shutdown_connections 2024-03-08T06:03:45.867 INFO:teuthology.orchestra.run.smithi053.stderr:2024-03-08T06:03:45.866+0000 7fbb53e00640 1 -- 172.21.15.53:0/3378624189 wait complete. 2024-03-08T06:03:45.882 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-08T06:03:45.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-08T06:03:45.888 INFO:tasks.ceph.osd.0:Started 2024-03-08T06:03:45.888 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-08T06:03:45.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-08T06:03:45.894 INFO:tasks.ceph.osd.1:Started 2024-03-08T06:03:45.894 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-08T06:03:45.894 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-08T06:03:45.899 INFO:tasks.ceph.osd.2:Started 2024-03-08T06:03:45.899 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-08T06:03:45.899 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-08T06:03:45.904 INFO:tasks.ceph.osd.3:Started 2024-03-08T06:03:45.904 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-08T06:03:45.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-08T06:03:45.911 INFO:tasks.ceph.osd.4:Started 2024-03-08T06:03:45.911 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-08T06:03:45.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-08T06:03:45.919 INFO:tasks.ceph.osd.5:Started 2024-03-08T06:03:45.919 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-08T06:03:45.919 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-08T06:03:45.926 INFO:tasks.ceph.osd.6:Started 2024-03-08T06:03:45.926 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-08T06:03:45.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-08T06:03:45.935 INFO:tasks.ceph.osd.7:Started 2024-03-08T06:03:45.936 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:03:45.943 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:45.943+0000 7fdee322b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T06:03:46.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.157+0000 7f9652600640 1 Processor -- start 2024-03-08T06:03:46.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.157+0000 7f9652600640 1 -- start start 2024-03-08T06:03:46.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.158+0000 7f9652600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f964c0cf190 0x7f964c161a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:46.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.158+0000 7f9652600640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f964c0b9790 0x7f964c0d2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:46.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.158+0000 7f9652600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964c162100 con 0x7f964c15df30 2024-03-08T06:03:46.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.158+0000 7f9652600640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964c1629a0 con 0x7f964c0b9790 2024-03-08T06:03:46.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.158+0000 7f9652600640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964c1623e0 con 0x7f964c0cf190 2024-03-08T06:03:46.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.159+0000 7f964be00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f964c0b9790 0x7f964c0d2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:46.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.159+0000 7f964be00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f964c0b9790 0x7f964c0d2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60486/0 (socket says 172.21.15.40:60486) 2024-03-08T06:03:46.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.159+0000 7f964be00640 1 -- 172.21.15.40:0/1018180020 learned_addr learned my addr 172.21.15.40:0/1018180020 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:46.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.159+0000 7f9643e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f964c0cf190 0x7f964c161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:46.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.159+0000 7f964be00640 1 -- 172.21.15.40:0/1018180020 >> v2:172.21.15.53:3300/0 conn(0x7f964c0cf190 msgr2=0x7f964c161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:46.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.159+0000 7f964be00640 1 --2- 172.21.15.40:0/1018180020 >> v2:172.21.15.53:3300/0 conn(0x7f964c0cf190 0x7f964c161a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:46.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.159+0000 7f964be00640 1 -- 172.21.15.40:0/1018180020 >> v1:172.21.15.53:6789/0 conn(0x7f964c15df30 legacy=0x7f964c0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:46.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.159+0000 7f964be00640 1 -- 172.21.15.40:0/1018180020 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c1626c0 con 0x7f964c0b9790 2024-03-08T06:03:46.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.160+0000 7f964be00640 1 --2- 172.21.15.40:0/1018180020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964c0b9790 0x7f964c0d2740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f964c08cb00 tx=0x7f964402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fee38d821f5a7ad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:46.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.161+0000 7f964aa00640 1 -- 172.21.15.40:0/1018180020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964402ca80 con 0x7f964c0b9790 2024-03-08T06:03:46.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.161+0000 7f964aa00640 1 -- 172.21.15.40:0/1018180020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964402cc20 con 0x7f964c0b9790 2024-03-08T06:03:46.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.161+0000 7f964aa00640 1 -- 172.21.15.40:0/1018180020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96440343f0 con 0x7f964c0b9790 2024-03-08T06:03:46.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.161+0000 7f9652600640 1 -- 172.21.15.40:0/1018180020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964c0b9790 msgr2=0x7f964c0d2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:46.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.161+0000 7f9652600640 1 --2- 172.21.15.40:0/1018180020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964c0b9790 0x7f964c0d2740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f964c08cb00 tx=0x7f964402bee0 comp rx=0 tx=0).stop 2024-03-08T06:03:46.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.161+0000 7f9652600640 1 -- 172.21.15.40:0/1018180020 shutdown_connections 2024-03-08T06:03:46.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.161+0000 7f9652600640 1 --2- 172.21.15.40:0/1018180020 >> v2:172.21.15.53:3300/0 conn(0x7f964c0cf190 0x7f964c161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:46.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.161+0000 7f9652600640 1 --2- 172.21.15.40:0/1018180020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964c0b9790 0x7f964c0d2740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:46.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.161+0000 7f9652600640 1 -- 172.21.15.40:0/1018180020 >> 172.21.15.40:0/1018180020 conn(0x7f964c06c600 msgr2=0x7f964c06ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:46.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.161+0000 7f9652600640 1 -- 172.21.15.40:0/1018180020 shutdown_connections 2024-03-08T06:03:46.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.161+0000 7f9652600640 1 -- 172.21.15.40:0/1018180020 wait complete. 2024-03-08T06:03:46.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.163+0000 7f9652600640 1 Processor -- start 2024-03-08T06:03:46.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.163+0000 7f9652600640 1 -- start start 2024-03-08T06:03:46.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.164+0000 7f9652600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f964c0b9790 0x7f964c160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:46.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.164+0000 7f9652600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964c0cf190 0x7f964c14a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:46.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.164+0000 7f9652600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f964c15df30 0x7f964c0c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:46.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.164+0000 7f9652600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c0b9bc0 con 0x7f964c0cf190 2024-03-08T06:03:46.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.164+0000 7f9652600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c15e310 con 0x7f964c0b9790 2024-03-08T06:03:46.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.164+0000 7f9652600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c0d2c80 con 0x7f964c15df30 2024-03-08T06:03:46.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.164+0000 7f9643e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964c0cf190 0x7f964c14a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.164+0000 7f9643e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964c0cf190 0x7f964c14a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60492/0 (socket says 172.21.15.40:60492) 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.164+0000 7f9643e00640 1 -- 172.21.15.40:0/3962892229 learned_addr learned my addr 172.21.15.40:0/3962892229 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.164+0000 7f964b400640 1 --2- 172.21.15.40:0/3962892229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f964c0b9790 0x7f964c160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.164+0000 7f964be00640 1 --2- 172.21.15.40:0/3962892229 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f964c15df30 0x7f964c0c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.165+0000 7f9643e00640 1 -- 172.21.15.40:0/3962892229 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f964c15df30 msgr2=0x7f964c0c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.165+0000 7f9643e00640 1 --2- 172.21.15.40:0/3962892229 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f964c15df30 0x7f964c0c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.165+0000 7f9643e00640 1 -- 172.21.15.40:0/3962892229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f964c0b9790 msgr2=0x7f964c160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.165+0000 7f9643e00640 1 --2- 172.21.15.40:0/3962892229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f964c0b9790 0x7f964c160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.165+0000 7f9643e00640 1 -- 172.21.15.40:0/3962892229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c161310 con 0x7f964c0cf190 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.165+0000 7f9643e00640 1 --2- 172.21.15.40:0/3962892229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964c0cf190 0x7f964c14a240 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f963400e9e0 tx=0x7f963400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.165+0000 7f9643400640 1 -- 172.21.15.40:0/3962892229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634017070 con 0x7f964c0cf190 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.165+0000 7f9643400640 1 -- 172.21.15.40:0/3962892229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963400c9d0 con 0x7f964c0cf190 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.165+0000 7f9643400640 1 -- 172.21.15.40:0/3962892229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963400cdc0 con 0x7f964c0cf190 2024-03-08T06:03:46.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.165+0000 7f9652600640 1 -- 172.21.15.40:0/3962892229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c0c8e80 con 0x7f964c0cf190 2024-03-08T06:03:46.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.166+0000 7f9652600640 1 -- 172.21.15.40:0/3962892229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c0c9440 con 0x7f964c0cf190 2024-03-08T06:03:46.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.166+0000 7f9643400640 1 -- 172.21.15.40:0/3962892229 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f963401a070 con 0x7f964c0cf190 2024-03-08T06:03:46.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.166+0000 7f9643400640 1 -- 172.21.15.40:0/3962892229 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f96340066b0 con 0x7f964c0cf190 2024-03-08T06:03:46.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.167+0000 7f9652600640 1 -- 172.21.15.40:0/3962892229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c067cb0 con 0x7f964c0cf190 2024-03-08T06:03:46.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.170+0000 7f9643400640 1 -- 172.21.15.40:0/3962892229 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9634005990 con 0x7f964c0cf190 2024-03-08T06:03:46.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.236+0000 7f9652600640 1 -- 172.21.15.40:0/3962892229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f964c0c7580 con 0x7f964c0cf190 2024-03-08T06:03:46.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.237+0000 7f9643400640 1 -- 172.21.15.40:0/3962892229 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f9634005b70 con 0x7f964c0cf190 2024-03-08T06:03:46.237 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:46.238 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:03:45.860054+0000","last_up_change":"0.000000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:03:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.238+0000 7f9652600640 1 -- 172.21.15.40:0/3962892229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964c0cf190 msgr2=0x7f964c14a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.238+0000 7f9652600640 1 --2- 172.21.15.40:0/3962892229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964c0cf190 0x7f964c14a240 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f963400e9e0 tx=0x7f963400eea0 comp rx=0 tx=0).stop 2024-03-08T06:03:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.238+0000 7f9652600640 1 -- 172.21.15.40:0/3962892229 shutdown_connections 2024-03-08T06:03:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.238+0000 7f9652600640 1 --2- 172.21.15.40:0/3962892229 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f964c15df30 0x7f964c0c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.238+0000 7f9652600640 1 --2- 172.21.15.40:0/3962892229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f964c0cf190 0x7f964c14a240 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.238+0000 7f9652600640 1 --2- 172.21.15.40:0/3962892229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f964c0b9790 0x7f964c160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.238+0000 7f9652600640 1 -- 172.21.15.40:0/3962892229 >> 172.21.15.40:0/3962892229 conn(0x7f964c06c600 msgr2=0x7f964c15b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.238+0000 7f9652600640 1 -- 172.21.15.40:0/3962892229 shutdown_connections 2024-03-08T06:03:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:46.238+0000 7f9652600640 1 -- 172.21.15.40:0/3962892229 wait complete. 2024-03-08T06:03:46.254 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-08T06:03:46.254 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-08T06:03:46.608 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-03-08T06:03:46.608+0000 7f88442c1640 -1 Falling back to public interface 2024-03-08T06:03:46.615 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-03-08T06:03:46.615+0000 7f7c669ff640 -1 Falling back to public interface 2024-03-08T06:03:46.618 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-03-08T06:03:46.618+0000 7fccdb80e640 -1 Falling back to public interface 2024-03-08T06:03:46.621 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-03-08T06:03:46.621+0000 7f6e15321640 -1 Falling back to public interface 2024-03-08T06:03:46.625 INFO:tasks.ceph.osd.7.smithi053.stderr:2024-03-08T06:03:46.624+0000 7f2d46489640 -1 Falling back to public interface 2024-03-08T06:03:46.625 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-03-08T06:03:46.625+0000 7f2efe72a640 -1 Falling back to public interface 2024-03-08T06:03:46.635 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-03-08T06:03:46.635+0000 7fc276f36640 -1 Falling back to public interface 2024-03-08T06:03:46.659 INFO:tasks.ceph.osd.6.smithi053.stderr:2024-03-08T06:03:46.659+0000 7fd56e3ac640 -1 Falling back to public interface 2024-03-08T06:03:46.927 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:46.928+0000 7f5845a42040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T06:03:47.044 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:47.043+0000 7fdee322b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T06:03:47.060 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-08T06:03:47.216 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.216+0000 7f1296c00640 1 Processor -- start 2024-03-08T06:03:47.216 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.216+0000 7f1296c00640 1 -- start start 2024-03-08T06:03:47.218 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.217+0000 7f1296c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f129015e140 0x7f12900d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:47.218 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.217+0000 7f1296c00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12900d4790 0x7f12900d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:47.218 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.217+0000 7f1296c00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12900b0940 con 0x7f129015e520 2024-03-08T06:03:47.219 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.217+0000 7f1296c00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12900cf6e0 con 0x7f129015e140 2024-03-08T06:03:47.219 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.217+0000 7f1296c00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12900cf410 con 0x7f12900d4790 2024-03-08T06:03:47.219 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.217+0000 7f128fe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f129015e140 0x7f12900d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:47.219 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.218+0000 7f128fe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f129015e140 0x7f12900d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41022/0 (socket says 172.21.15.40:41022) 2024-03-08T06:03:47.219 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.218+0000 7f128fe00640 1 -- 172.21.15.40:0/861298000 learned_addr learned my addr 172.21.15.40:0/861298000 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:47.219 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.218+0000 7f1294c00640 1 --1- 172.21.15.40:0/861298000 >> v1:172.21.15.40:6789/0 conn(0x7f129015e520 0x7f12900d1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33142/0 (socket says 172.21.15.40:33142) 2024-03-08T06:03:47.219 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.218+0000 7f128f400640 1 --2- 172.21.15.40:0/861298000 >> v2:172.21.15.53:3300/0 conn(0x7f12900d4790 0x7f12900d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:47.219 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.218+0000 7f128fe00640 1 -- 172.21.15.40:0/861298000 >> v2:172.21.15.53:3300/0 conn(0x7f12900d4790 msgr2=0x7f12900d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:47.219 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.218+0000 7f128fe00640 1 --2- 172.21.15.40:0/861298000 >> v2:172.21.15.53:3300/0 conn(0x7f12900d4790 0x7f12900d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:47.219 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.218+0000 7f128fe00640 1 -- 172.21.15.40:0/861298000 >> v1:172.21.15.40:6789/0 conn(0x7f129015e520 legacy=0x7f12900d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:47.219 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.218+0000 7f128fe00640 1 -- 172.21.15.40:0/861298000 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12900cf9a0 con 0x7f129015e140 2024-03-08T06:03:47.221 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.219+0000 7f128fe00640 1 --2- 172.21.15.40:0/861298000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f129015e140 0x7f12900d03c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12900d1750 tx=0x7f128002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db3d7173d0eaa6b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:47.221 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.219+0000 7f128ea00640 1 -- 172.21.15.40:0/861298000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280035020 con 0x7f129015e140 2024-03-08T06:03:47.221 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.219+0000 7f128ea00640 1 -- 172.21.15.40:0/861298000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1280031870 con 0x7f129015e140 2024-03-08T06:03:47.221 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.219+0000 7f128ea00640 1 -- 172.21.15.40:0/861298000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280031c40 con 0x7f129015e140 2024-03-08T06:03:47.221 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.220+0000 7f1296c00640 1 -- 172.21.15.40:0/861298000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f129015e140 msgr2=0x7f12900d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:47.221 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.220+0000 7f1296c00640 1 --2- 172.21.15.40:0/861298000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f129015e140 0x7f12900d03c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12900d1750 tx=0x7f128002b910 comp rx=0 tx=0).stop 2024-03-08T06:03:47.222 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.220+0000 7f1296c00640 1 -- 172.21.15.40:0/861298000 shutdown_connections 2024-03-08T06:03:47.222 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.220+0000 7f1296c00640 1 --2- 172.21.15.40:0/861298000 >> v2:172.21.15.53:3300/0 conn(0x7f12900d4790 0x7f12900d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:47.222 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.220+0000 7f1296c00640 1 --2- 172.21.15.40:0/861298000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f129015e140 0x7f12900d03c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:47.222 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.220+0000 7f1296c00640 1 -- 172.21.15.40:0/861298000 >> 172.21.15.40:0/861298000 conn(0x7f12900d3c70 msgr2=0x7f12900d4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:47.222 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.220+0000 7f1296c00640 1 -- 172.21.15.40:0/861298000 shutdown_connections 2024-03-08T06:03:47.222 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.220+0000 7f1296c00640 1 -- 172.21.15.40:0/861298000 wait complete. 2024-03-08T06:03:47.225 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.222+0000 7f1296c00640 1 Processor -- start 2024-03-08T06:03:47.225 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.222+0000 7f1296c00640 1 -- start start 2024-03-08T06:03:47.225 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.223+0000 7f1296c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f12900d4790 0x7f12900b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:47.225 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.223+0000 7f1296c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f129015e140 0x7f12900b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:47.225 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.223+0000 7f1296c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f129015e520 0x7f12900b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:47.225 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.223+0000 7f1296c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12900b1fb0 con 0x7f129015e140 2024-03-08T06:03:47.225 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.223+0000 7f1296c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12900b1e30 con 0x7f129015e520 2024-03-08T06:03:47.225 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.223+0000 7f1296c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12900b2130 con 0x7f12900d4790 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.223+0000 7f128f400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f129015e140 0x7f12900b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.223+0000 7f128f400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f129015e140 0x7f12900b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41034/0 (socket says 172.21.15.40:41034) 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.223+0000 7f128f400640 1 -- 172.21.15.40:0/2662631706 learned_addr learned my addr 172.21.15.40:0/2662631706 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.223+0000 7f1294c00640 1 --2- 172.21.15.40:0/2662631706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f129015e520 0x7f12900b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.223+0000 7f128fe00640 1 --2- 172.21.15.40:0/2662631706 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f12900d4790 0x7f12900b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.224+0000 7f128f400640 1 -- 172.21.15.40:0/2662631706 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f12900d4790 msgr2=0x7f12900b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.224+0000 7f128f400640 1 --2- 172.21.15.40:0/2662631706 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f12900d4790 0x7f12900b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.224+0000 7f128f400640 1 -- 172.21.15.40:0/2662631706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f129015e520 msgr2=0x7f12900b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.224+0000 7f128f400640 1 --2- 172.21.15.40:0/2662631706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f129015e520 0x7f12900b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.224+0000 7f128f400640 1 -- 172.21.15.40:0/2662631706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12900b3b20 con 0x7f129015e140 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.224+0000 7f128f400640 1 --2- 172.21.15.40:0/2662631706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f129015e140 0x7f12900b3220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e9b0 tx=0x7f127c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:47.226 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.224+0000 7f1287e00640 1 -- 172.21.15.40:0/2662631706 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c017070 con 0x7f129015e140 2024-03-08T06:03:47.227 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.224+0000 7f1287e00640 1 -- 172.21.15.40:0/2662631706 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c00c9a0 con 0x7f129015e140 2024-03-08T06:03:47.227 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.224+0000 7f1296c00640 1 -- 172.21.15.40:0/2662631706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12900b3e00 con 0x7f129015e140 2024-03-08T06:03:47.227 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.224+0000 7f1287e00640 1 -- 172.21.15.40:0/2662631706 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c00cd90 con 0x7f129015e140 2024-03-08T06:03:47.227 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.224+0000 7f1296c00640 1 -- 172.21.15.40:0/2662631706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12900cb5f0 con 0x7f129015e140 2024-03-08T06:03:47.227 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.225+0000 7f1287e00640 1 -- 172.21.15.40:0/2662631706 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f127c01a070 con 0x7f129015e140 2024-03-08T06:03:47.227 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.225+0000 7f1287e00640 1 -- 172.21.15.40:0/2662631706 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f127c01cc10 con 0x7f129015e140 2024-03-08T06:03:47.227 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.225+0000 7f1296c00640 1 -- 172.21.15.40:0/2662631706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1290075160 con 0x7f129015e140 2024-03-08T06:03:47.228 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.228+0000 7f1287e00640 1 -- 172.21.15.40:0/2662631706 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f127c01c540 con 0x7f129015e140 2024-03-08T06:03:47.259 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:47.259+0000 7f5845a42040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T06:03:47.293 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.293+0000 7f1296c00640 1 -- 172.21.15.40:0/2662631706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f129015fdd0 con 0x7f129015e140 2024-03-08T06:03:47.294 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.294+0000 7f1287e00640 1 -- 172.21.15.40:0/2662631706 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f127c01c7e0 con 0x7f129015e140 2024-03-08T06:03:47.295 INFO:teuthology.misc.health.smithi040.stdout: 2024-03-08T06:03:47.295 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":10,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:03:45.860054+0000","last_up_change":"0.000000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:03:47.296 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.295+0000 7f1296c00640 1 -- 172.21.15.40:0/2662631706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f129015e140 msgr2=0x7f12900b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:47.296 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.295+0000 7f1296c00640 1 --2- 172.21.15.40:0/2662631706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f129015e140 0x7f12900b3220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e9b0 tx=0x7f127c00ee70 comp rx=0 tx=0).stop 2024-03-08T06:03:47.296 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.295+0000 7f1296c00640 1 -- 172.21.15.40:0/2662631706 shutdown_connections 2024-03-08T06:03:47.297 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.295+0000 7f1296c00640 1 --2- 172.21.15.40:0/2662631706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f129015e520 0x7f12900b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:47.297 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.295+0000 7f1296c00640 1 --2- 172.21.15.40:0/2662631706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f129015e140 0x7f12900b3220 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:47.297 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.295+0000 7f1296c00640 1 --2- 172.21.15.40:0/2662631706 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f12900d4790 0x7f12900b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:47.297 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.295+0000 7f1296c00640 1 -- 172.21.15.40:0/2662631706 >> 172.21.15.40:0/2662631706 conn(0x7f12900d3c70 msgr2=0x7f129015c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:47.297 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.296+0000 7f1296c00640 1 -- 172.21.15.40:0/2662631706 shutdown_connections 2024-03-08T06:03:47.297 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:47.296+0000 7f1296c00640 1 -- 172.21.15.40:0/2662631706 wait complete. 2024-03-08T06:03:47.309 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-08T06:03:47.323 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:47.323+0000 7f5845a42040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T06:03:47.385 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:47.384+0000 7fdee322b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T06:03:47.449 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:47.448+0000 7fdee322b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T06:03:47.460 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:47.460+0000 7f5845a42040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T06:03:47.593 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-03-08T06:03:47.593+0000 7fdee322b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T06:03:48.505 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:48.505+0000 7f57e6200640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-08T06:03:48.507 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-03-08T06:03:48.508+0000 7f57e6200640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-08T06:03:50.184 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-03-08T06:03:50.184+0000 7f2efe72a640 -1 osd.2 0 log_to_monitors true 2024-03-08T06:03:50.204 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-03-08T06:03:50.203+0000 7fc276f36640 -1 osd.4 0 log_to_monitors true 2024-03-08T06:03:50.204 INFO:tasks.ceph.osd.7.smithi053.stderr:2024-03-08T06:03:50.204+0000 7f2d46489640 -1 osd.7 0 log_to_monitors true 2024-03-08T06:03:50.208 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-03-08T06:03:50.208+0000 7f6e15321640 -1 osd.3 0 log_to_monitors true 2024-03-08T06:03:50.392 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-03-08T06:03:50.392+0000 7fccdb80e640 -1 osd.5 0 log_to_monitors true 2024-03-08T06:03:50.467 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-03-08T06:03:50.467+0000 7f7c669ff640 -1 osd.1 0 log_to_monitors true 2024-03-08T06:03:50.468 INFO:tasks.ceph.osd.6.smithi053.stderr:2024-03-08T06:03:50.467+0000 7fd56e3ac640 -1 osd.6 0 log_to_monitors true 2024-03-08T06:03:50.478 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-03-08T06:03:50.478+0000 7f88442c1640 -1 osd.0 0 log_to_monitors true 2024-03-08T06:03:53.213 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-03-08T06:03:53.210+0000 7fccd5600640 -1 osd.5 0 waiting for initial osdmap 2024-03-08T06:03:53.216 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-03-08T06:03:53.215+0000 7fcccc000640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:03:53.242 INFO:tasks.ceph.osd.7.smithi053.stderr:2024-03-08T06:03:53.242+0000 7f2d42800640 -1 osd.7 0 waiting for initial osdmap 2024-03-08T06:03:53.248 INFO:tasks.ceph.osd.7.smithi053.stderr:2024-03-08T06:03:53.248+0000 7f2d37400640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:03:53.264 INFO:tasks.ceph.osd.6.smithi053.stderr:2024-03-08T06:03:53.264+0000 7fd568800640 -1 osd.6 0 waiting for initial osdmap 2024-03-08T06:03:53.272 INFO:tasks.ceph.osd.6.smithi053.stderr:2024-03-08T06:03:53.271+0000 7fd55f200640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:03:53.286 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-03-08T06:03:53.285+0000 7fc271400640 -1 osd.4 0 waiting for initial osdmap 2024-03-08T06:03:53.292 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-03-08T06:03:53.291+0000 7fc267e00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:03:53.740 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-03-08T06:03:53.740+0000 7f883e800640 -1 osd.0 0 waiting for initial osdmap 2024-03-08T06:03:53.747 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-03-08T06:03:53.747+0000 7f8835200640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:03:53.807 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-03-08T06:03:53.807+0000 7f2ef8c00640 -1 osd.2 0 waiting for initial osdmap 2024-03-08T06:03:53.807 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-03-08T06:03:53.807+0000 7f7c60e00640 -1 osd.1 0 waiting for initial osdmap 2024-03-08T06:03:53.815 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-03-08T06:03:53.815+0000 7f7c57800640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:03:53.819 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-03-08T06:03:53.819+0000 7f2eef600640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:03:53.891 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-03-08T06:03:53.891+0000 7f6e0f800640 -1 osd.3 0 waiting for initial osdmap 2024-03-08T06:03:53.901 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-03-08T06:03:53.901+0000 7f6e06200640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T06:03:54.115 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-08T06:03:54.318 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.318+0000 7f16da400640 1 Processor -- start 2024-03-08T06:03:54.318 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.319+0000 7f16da400640 1 -- start start 2024-03-08T06:03:54.320 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.320+0000 7f16da400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f16d40aee40 0x7f16d40ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.320+0000 7f16da400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f16d4070380 0x7f16d406c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.320+0000 7f16da400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d40cc110 con 0x7f16d4071b40 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.320+0000 7f16da400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d40b5400 con 0x7f16d4070380 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.320+0000 7f16da400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40c9fa0 con 0x7f16d40aee40 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.320+0000 7f16d3400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f16d4071b40 0x7f16d4071f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33146/0 (socket says 172.21.15.40:33146) 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.320+0000 7f16d3400640 1 -- 172.21.15.40:0/2042418951 learned_addr learned my addr 172.21.15.40:0/2042418951 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.320+0000 7f16d3e00640 1 --2- 172.21.15.40:0/2042418951 >> v2:172.21.15.40:3300/0 conn(0x7f16d4070380 0x7f16d406c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.320+0000 7f16caa00640 1 --2- 172.21.15.40:0/2042418951 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f16d40aee40 0x7f16d40ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.320+0000 7f16d2a00640 1 -- 172.21.15.40:0/2042418951 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14394383 0 0) 0x7f16d40cc110 con 0x7f16d4071b40 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.321+0000 7f16d2a00640 1 -- 172.21.15.40:0/2042418951 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16b8003520 con 0x7f16d4071b40 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.321+0000 7f16d3e00640 1 -- 172.21.15.40:0/2042418951 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f16d40aee40 msgr2=0x7f16d40ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.321+0000 7f16d3e00640 1 --2- 172.21.15.40:0/2042418951 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f16d40aee40 0x7f16d40ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.321 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.321+0000 7f16d3e00640 1 -- 172.21.15.40:0/2042418951 >> v1:172.21.15.40:6789/0 conn(0x7f16d4071b40 legacy=0x7f16d4071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.322 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.321+0000 7f16d3e00640 1 -- 172.21.15.40:0/2042418951 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d40ca280 con 0x7f16d4070380 2024-03-08T06:03:54.323 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.322+0000 7f16d3e00640 1 --2- 172.21.15.40:0/2042418951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16d4070380 0x7f16d406c3e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009d90 tx=0x7f16cc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fea5bd366c47da server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:54.323 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.322+0000 7f16d2a00640 1 -- 172.21.15.40:0/2042418951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc039070 con 0x7f16d4070380 2024-03-08T06:03:54.323 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.322+0000 7f16d2a00640 1 -- 172.21.15.40:0/2042418951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc02cc50 con 0x7f16d4070380 2024-03-08T06:03:54.323 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.322+0000 7f16d2a00640 1 -- 172.21.15.40:0/2042418951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc034440 con 0x7f16d4070380 2024-03-08T06:03:54.323 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.323+0000 7f16da400640 1 -- 172.21.15.40:0/2042418951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16d4070380 msgr2=0x7f16d406c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.323 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.323+0000 7f16da400640 1 --2- 172.21.15.40:0/2042418951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16d4070380 0x7f16d406c3e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009d90 tx=0x7f16cc02bea0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.323 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.323+0000 7f16da400640 1 -- 172.21.15.40:0/2042418951 shutdown_connections 2024-03-08T06:03:54.323 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.323+0000 7f16da400640 1 --2- 172.21.15.40:0/2042418951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16d4070380 0x7f16d406c3e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.323 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.323+0000 7f16da400640 1 --2- 172.21.15.40:0/2042418951 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f16d40aee40 0x7f16d40ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.323 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.323+0000 7f16da400640 1 -- 172.21.15.40:0/2042418951 >> 172.21.15.40:0/2042418951 conn(0x7f16d40cf8b0 msgr2=0x7f16d40cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:54.323 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.323+0000 7f16da400640 1 -- 172.21.15.40:0/2042418951 shutdown_connections 2024-03-08T06:03:54.324 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.323+0000 7f16da400640 1 -- 172.21.15.40:0/2042418951 wait complete. 2024-03-08T06:03:54.324 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.324+0000 7f16da400640 1 Processor -- start 2024-03-08T06:03:54.324 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.325+0000 7f16da400640 1 -- start start 2024-03-08T06:03:54.326 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.325+0000 7f16da400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f16d4070380 0x7f16d40c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.326 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.325+0000 7f16da400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d4071b40 0x7f16d40cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.326 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.325+0000 7f16da400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16d40aee40 0x7f16d41f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.326 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.325+0000 7f16da400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40d0550 con 0x7f16d40aee40 2024-03-08T06:03:54.326 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.325+0000 7f16da400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40d03d0 con 0x7f16d4071b40 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.325+0000 7f16da400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d415bfa0 con 0x7f16d4070380 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.326+0000 7f16d3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16d40aee40 0x7f16d41f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.326+0000 7f16d3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16d40aee40 0x7f16d41f6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41164/0 (socket says 172.21.15.40:41164) 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.326+0000 7f16d3e00640 1 -- 172.21.15.40:0/2514750724 learned_addr learned my addr 172.21.15.40:0/2514750724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.326+0000 7f16d3400640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f16d4070380 0x7f16d40c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.326+0000 7f16caa00640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d4071b40 0x7f16d40cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.326+0000 7f16d3e00640 1 -- 172.21.15.40:0/2514750724 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f16d4070380 msgr2=0x7f16d40c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.326+0000 7f16d3e00640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f16d4070380 0x7f16d40c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.326+0000 7f16d3e00640 1 -- 172.21.15.40:0/2514750724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d4071b40 msgr2=0x7f16d40cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.326+0000 7f16d3e00640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d4071b40 0x7f16d40cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.326+0000 7f16d3e00640 1 -- 172.21.15.40:0/2514750724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d40cdf20 con 0x7f16d40aee40 2024-03-08T06:03:54.327 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.326+0000 7f16d3e00640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16d40aee40 0x7f16d41f6d70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009d90 tx=0x7f16cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:54.328 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.327+0000 7f16cbe00640 1 -- 172.21.15.40:0/2514750724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc039050 con 0x7f16d40aee40 2024-03-08T06:03:54.328 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.327+0000 7f16cbe00640 1 -- 172.21.15.40:0/2514750724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc045070 con 0x7f16d40aee40 2024-03-08T06:03:54.328 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.327+0000 7f16cbe00640 1 -- 172.21.15.40:0/2514750724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc009450 con 0x7f16d40aee40 2024-03-08T06:03:54.328 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.327+0000 7f16da400640 1 -- 172.21.15.40:0/2514750724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d40ce200 con 0x7f16d40aee40 2024-03-08T06:03:54.328 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.327+0000 7f16da400640 1 -- 172.21.15.40:0/2514750724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d40ce7c0 con 0x7f16d40aee40 2024-03-08T06:03:54.328 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.328+0000 7f16da400640 1 -- 172.21.15.40:0/2514750724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d4075160 con 0x7f16d40aee40 2024-03-08T06:03:54.329 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.329+0000 7f16cbe00640 1 -- 172.21.15.40:0/2514750724 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f16cc03e610 con 0x7f16d40aee40 2024-03-08T06:03:54.330 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.330+0000 7f16cbe00640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f16b006b440 0x7f16b006d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.330 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.330+0000 7f16cbe00640 1 -- 172.21.15.40:0/2514750724 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f16cc0ac7b0 con 0x7f16d40aee40 2024-03-08T06:03:54.330 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.330+0000 7f16d3400640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f16b006b440 0x7f16b006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.330 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.331+0000 7f16d3400640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f16b006b440 0x7f16b006d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f16c4005880 tx=0x7f16c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:54.335 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.335+0000 7f16cbe00640 1 -- 172.21.15.40:0/2514750724 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16cc07bab0 con 0x7f16d40aee40 2024-03-08T06:03:54.529 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.528+0000 7f16da400640 1 -- 172.21.15.40:0/2514750724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16d4074570 con 0x7f16d40aee40 2024-03-08T06:03:54.529 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.530+0000 7f16cbe00640 1 -- 172.21.15.40:0/2514750724 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10825 (secure 0 0 0) 0x7f16cc07b3b0 con 0x7f16d40aee40 2024-03-08T06:03:54.530 INFO:teuthology.misc.health.smithi040.stdout: 2024-03-08T06:03:54.530 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":13,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:03:54.205597+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:03:54.533 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 -- 172.21.15.40:0/2514750724 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f16b006b440 msgr2=0x7f16b006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.533 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f16b006b440 0x7f16b006d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f16c4005880 tx=0x7f16c4009040 comp rx=0 tx=0).stop 2024-03-08T06:03:54.533 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 -- 172.21.15.40:0/2514750724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16d40aee40 msgr2=0x7f16d41f6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.533 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16d40aee40 0x7f16d41f6d70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009d90 tx=0x7f16cc004a40 comp rx=0 tx=0).stop 2024-03-08T06:03:54.533 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 -- 172.21.15.40:0/2514750724 shutdown_connections 2024-03-08T06:03:54.533 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f16b006b440 0x7f16b006d900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.533 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16d40aee40 0x7f16d41f6d70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.533 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16d4071b40 0x7f16d40cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.534 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 --2- 172.21.15.40:0/2514750724 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f16d4070380 0x7f16d40c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.534 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 -- 172.21.15.40:0/2514750724 >> 172.21.15.40:0/2514750724 conn(0x7f16d40cf8b0 msgr2=0x7f16d4071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:54.534 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 -- 172.21.15.40:0/2514750724 shutdown_connections 2024-03-08T06:03:54.534 INFO:teuthology.misc.health.smithi040.stderr:2024-03-08T06:03:54.533+0000 7f16da400640 1 -- 172.21.15.40:0/2514750724 wait complete. 2024-03-08T06:03:54.547 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-08T06:03:54.547 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-08T06:03:54.547 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-08T06:03:54.547 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-08T06:03:54.552 INFO:tasks.ceph.mds.a:Started 2024-03-08T06:03:54.552 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-08T06:03:54.552 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-08T06:03:54.559 INFO:tasks.ceph.mds.c:Started 2024-03-08T06:03:54.559 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-08T06:03:54.559 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-08T06:03:54.567 INFO:tasks.ceph.mds.b:Started 2024-03-08T06:03:54.567 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-08T06:03:54.567 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-08T06:03:54.574 INFO:tasks.ceph.mds.d:Started 2024-03-08T06:03:54.574 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-08T06:03:54.575 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:03:54.695 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-03-08T06:03:54.703 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-03-08T06:03:54.727 INFO:tasks.ceph.mds.b.smithi053.stdout:starting mds.b at 2024-03-08T06:03:54.727 INFO:tasks.ceph.mds.d.smithi053.stdout:starting mds.d at 2024-03-08T06:03:54.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.755+0000 7f914f200640 1 Processor -- start 2024-03-08T06:03:54.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.756+0000 7f914f200640 1 -- start start 2024-03-08T06:03:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914f200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f915015dce0 0x7f91500d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914f200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f915006c9a0 0x7f91500b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914f200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91500d25f0 con 0x7f91500d3690 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914f200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91500b9b80 con 0x7f915015dce0 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914f200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91500b7ca0 con 0x7f915006c9a0 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914d400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f91500d3690 0x7f915006c290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33176/0 (socket says 172.21.15.40:33176) 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914d400640 1 -- 172.21.15.40:0/2373576234 learned_addr learned my addr 172.21.15.40:0/2373576234 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914de00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f915015dce0 0x7f91500d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914e800640 1 --2- 172.21.15.40:0/2373576234 >> v2:172.21.15.53:3300/0 conn(0x7f915006c9a0 0x7f91500b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914ca00640 1 -- 172.21.15.40:0/2373576234 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999546866 0 0) 0x7f91500d25f0 con 0x7f91500d3690 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914ca00640 1 -- 172.21.15.40:0/2373576234 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9134003520 con 0x7f91500d3690 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.757+0000 7f914de00640 1 -- 172.21.15.40:0/2373576234 >> v2:172.21.15.53:3300/0 conn(0x7f915006c9a0 msgr2=0x7f91500b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.758+0000 7f914de00640 1 --2- 172.21.15.40:0/2373576234 >> v2:172.21.15.53:3300/0 conn(0x7f915006c9a0 0x7f91500b9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.758+0000 7f914de00640 1 -- 172.21.15.40:0/2373576234 >> v1:172.21.15.40:6789/0 conn(0x7f91500d3690 legacy=0x7f915006c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.758+0000 7f914de00640 1 -- 172.21.15.40:0/2373576234 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91500d28d0 con 0x7f915015dce0 2024-03-08T06:03:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.758+0000 7f914e800640 1 --2- 172.21.15.40:0/2373576234 >> v2:172.21.15.53:3300/0 conn(0x7f915006c9a0 0x7f91500b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:03:54.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.759+0000 7f914de00640 1 --2- 172.21.15.40:0/2373576234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915015dce0 0x7f91500d1fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f915008ce20 tx=0x7f914002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bc1c0ecabc83c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.759+0000 7f914ca00640 1 -- 172.21.15.40:0/2373576234 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140035020 con 0x7f915015dce0 2024-03-08T06:03:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.759+0000 7f914ca00640 1 -- 172.21.15.40:0/2373576234 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9140031830 con 0x7f915015dce0 2024-03-08T06:03:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.759+0000 7f914ca00640 1 -- 172.21.15.40:0/2373576234 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140031c00 con 0x7f915015dce0 2024-03-08T06:03:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.759+0000 7f914f200640 1 -- 172.21.15.40:0/2373576234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915015dce0 msgr2=0x7f91500d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.759+0000 7f914f200640 1 --2- 172.21.15.40:0/2373576234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915015dce0 0x7f91500d1fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f915008ce20 tx=0x7f914002b8d0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.759+0000 7f914f200640 1 -- 172.21.15.40:0/2373576234 shutdown_connections 2024-03-08T06:03:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.759+0000 7f914f200640 1 --2- 172.21.15.40:0/2373576234 >> v2:172.21.15.53:3300/0 conn(0x7f915006c9a0 0x7f91500b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.759+0000 7f914f200640 1 --2- 172.21.15.40:0/2373576234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915015dce0 0x7f91500d1fa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.759+0000 7f914f200640 1 -- 172.21.15.40:0/2373576234 >> 172.21.15.40:0/2373576234 conn(0x7f9150066fc0 msgr2=0x7f91500d3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.760+0000 7f914f200640 1 -- 172.21.15.40:0/2373576234 shutdown_connections 2024-03-08T06:03:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.760+0000 7f914f200640 1 -- 172.21.15.40:0/2373576234 wait complete. 2024-03-08T06:03:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.761+0000 7f914f200640 1 Processor -- start 2024-03-08T06:03:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.761+0000 7f914f200640 1 -- start start 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.762+0000 7f914f200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915006c9a0 0x7f91500c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.762+0000 7f914f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91500d3690 0x7f91501e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.762+0000 7f914f200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f915015dce0 0x7f91501e6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.762+0000 7f914f200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91500d3170 con 0x7f915006c9a0 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.762+0000 7f914f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91500d2e90 con 0x7f91500d3690 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.762+0000 7f914f200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915015e0c0 con 0x7f915015dce0 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.762+0000 7f914de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915006c9a0 0x7f91500c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.762+0000 7f914d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91500d3690 0x7f91501e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.762+0000 7f914de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915006c9a0 0x7f91500c94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41216/0 (socket says 172.21.15.40:41216) 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.762+0000 7f914de00640 1 -- 172.21.15.40:0/1052246285 learned_addr learned my addr 172.21.15.40:0/1052246285 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.762+0000 7f914e800640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f915015dce0 0x7f91501e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f914de00640 1 -- 172.21.15.40:0/1052246285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f915015dce0 msgr2=0x7f91501e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f914de00640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f915015dce0 0x7f91501e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f914de00640 1 -- 172.21.15.40:0/1052246285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91500d3690 msgr2=0x7f91501e5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f914de00640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91500d3690 0x7f91501e5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f914de00640 1 -- 172.21.15.40:0/1052246285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91501f4f20 con 0x7f915006c9a0 2024-03-08T06:03:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f914e800640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f915015dce0 0x7f91501e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:03:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f914d400640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91500d3690 0x7f91501e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:03:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f914de00640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915006c9a0 0x7f91500c94b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9140000c00 tx=0x7f91400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f9146000640 1 -- 172.21.15.40:0/1052246285 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140040070 con 0x7f915006c9a0 2024-03-08T06:03:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f9146000640 1 -- 172.21.15.40:0/1052246285 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914003a910 con 0x7f915006c9a0 2024-03-08T06:03:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f9146000640 1 -- 172.21.15.40:0/1052246285 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140035040 con 0x7f915006c9a0 2024-03-08T06:03:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.763+0000 7f914f200640 1 -- 172.21.15.40:0/1052246285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91501f5200 con 0x7f915006c9a0 2024-03-08T06:03:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.764+0000 7f914f200640 1 -- 172.21.15.40:0/1052246285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91501f57c0 con 0x7f915006c9a0 2024-03-08T06:03:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.765+0000 7f914f200640 1 -- 172.21.15.40:0/1052246285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91500d2690 con 0x7f915006c9a0 2024-03-08T06:03:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.765+0000 7f9146000640 1 -- 172.21.15.40:0/1052246285 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f9140038020 con 0x7f915006c9a0 2024-03-08T06:03:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.766+0000 7f9146000640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f913006b440 0x7f913006d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.766+0000 7f9146000640 1 -- 172.21.15.40:0/1052246285 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f91400ab320 con 0x7f915006c9a0 2024-03-08T06:03:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.767+0000 7f914d400640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f913006b440 0x7f913006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.767+0000 7f914d400640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f913006b440 0x7f913006d900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91501e6b90 tx=0x7f913c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.771+0000 7f9146000640 1 -- 172.21.15.40:0/1052246285 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91500d2690 con 0x7f915006c9a0 2024-03-08T06:03:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.964+0000 7f914f200640 1 -- 172.21.15.40:0/1052246285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f915006c500 con 0x7f915006c9a0 2024-03-08T06:03:54.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.965+0000 7f9146000640 1 -- 172.21.15.40:0/1052246285 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10825 (secure 0 0 0) 0x7f915006c500 con 0x7f915006c9a0 2024-03-08T06:03:54.966 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:54.966 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":13,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:03:54.205597+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:03:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 -- 172.21.15.40:0/1052246285 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f913006b440 msgr2=0x7f913006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f913006b440 0x7f913006d900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91501e6b90 tx=0x7f913c009040 comp rx=0 tx=0).stop 2024-03-08T06:03:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 -- 172.21.15.40:0/1052246285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915006c9a0 msgr2=0x7f91500c94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915006c9a0 0x7f91500c94b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9140000c00 tx=0x7f91400046c0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 -- 172.21.15.40:0/1052246285 shutdown_connections 2024-03-08T06:03:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f913006b440 0x7f913006d900 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f915015dce0 0x7f91501e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91500d3690 0x7f91501e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 --2- 172.21.15.40:0/1052246285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915006c9a0 0x7f91500c94b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 -- 172.21.15.40:0/1052246285 >> 172.21.15.40:0/1052246285 conn(0x7f9150066fc0 msgr2=0x7f91500cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 -- 172.21.15.40:0/1052246285 shutdown_connections 2024-03-08T06:03:54.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:54.968+0000 7f914f200640 1 -- 172.21.15.40:0/1052246285 wait complete. 2024-03-08T06:03:54.984 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-03-08T06:03:54.984 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:03:55.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.194+0000 7f9a1be00640 1 Processor -- start 2024-03-08T06:03:55.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.195+0000 7f9a1be00640 1 -- start start 2024-03-08T06:03:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.196+0000 7f9a1be00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c145520 0x7f9a1c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.196+0000 7f9a1be00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9a1c15bea0 0x7f9a1c0716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.196+0000 7f9a1be00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c15ceb0 con 0x7f9a1c0af040 2024-03-08T06:03:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.196+0000 7f9a1be00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c15c330 con 0x7f9a1c15bea0 2024-03-08T06:03:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.196+0000 7f9a1be00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c15c610 con 0x7f9a1c145520 2024-03-08T06:03:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.197+0000 7f9a13e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9a1c15bea0 0x7f9a1c0716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.197+0000 7f9a1b400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9a1c0af040 0x7f9a1c06d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33184/0 (socket says 172.21.15.40:33184) 2024-03-08T06:03:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.197+0000 7f9a1b400640 1 -- 172.21.15.40:0/4037853963 learned_addr learned my addr 172.21.15.40:0/4037853963 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:03:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.197+0000 7f9a1aa00640 1 --2- 172.21.15.40:0/4037853963 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c145520 0x7f9a1c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.197+0000 7f9a1a000640 1 -- 172.21.15.40:0/4037853963 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414476809 0 0) 0x7f9a1c15ceb0 con 0x7f9a1c0af040 2024-03-08T06:03:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.197+0000 7f9a1a000640 1 -- 172.21.15.40:0/4037853963 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a00003520 con 0x7f9a1c0af040 2024-03-08T06:03:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.197+0000 7f9a13e00640 1 -- 172.21.15.40:0/4037853963 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c145520 msgr2=0x7f9a1c15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.197+0000 7f9a13e00640 1 --2- 172.21.15.40:0/4037853963 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c145520 0x7f9a1c15d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.197+0000 7f9a13e00640 1 -- 172.21.15.40:0/4037853963 >> v1:172.21.15.40:6789/0 conn(0x7f9a1c0af040 legacy=0x7f9a1c06d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.197+0000 7f9a13e00640 1 -- 172.21.15.40:0/4037853963 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c15c8f0 con 0x7f9a1c15bea0 2024-03-08T06:03:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.198+0000 7f9a13e00640 1 --2- 172.21.15.40:0/4037853963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c15bea0 0x7f9a1c0716e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9a040097a0 tx=0x7f9a0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82480f5508af3ec0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.198+0000 7f9a1a000640 1 -- 172.21.15.40:0/4037853963 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04035020 con 0x7f9a1c15bea0 2024-03-08T06:03:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.199+0000 7f9a1a000640 1 -- 172.21.15.40:0/4037853963 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04031830 con 0x7f9a1c15bea0 2024-03-08T06:03:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.199+0000 7f9a1a000640 1 -- 172.21.15.40:0/4037853963 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04031be0 con 0x7f9a1c15bea0 2024-03-08T06:03:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.199+0000 7f9a1be00640 1 -- 172.21.15.40:0/4037853963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c15bea0 msgr2=0x7f9a1c0716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.199+0000 7f9a1be00640 1 --2- 172.21.15.40:0/4037853963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c15bea0 0x7f9a1c0716e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9a040097a0 tx=0x7f9a0402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:03:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.199+0000 7f9a1be00640 1 -- 172.21.15.40:0/4037853963 shutdown_connections 2024-03-08T06:03:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.199+0000 7f9a1be00640 1 --2- 172.21.15.40:0/4037853963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c15bea0 0x7f9a1c0716e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.199+0000 7f9a1be00640 1 --2- 172.21.15.40:0/4037853963 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c145520 0x7f9a1c15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.199+0000 7f9a1be00640 1 -- 172.21.15.40:0/4037853963 >> 172.21.15.40:0/4037853963 conn(0x7f9a1c0b1850 msgr2=0x7f9a1c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.199+0000 7f9a1be00640 1 -- 172.21.15.40:0/4037853963 shutdown_connections 2024-03-08T06:03:55.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.199+0000 7f9a1be00640 1 -- 172.21.15.40:0/4037853963 wait complete. 2024-03-08T06:03:55.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.201+0000 7f9a1be00640 1 Processor -- start 2024-03-08T06:03:55.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.201+0000 7f9a1be00640 1 -- start start 2024-03-08T06:03:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.202+0000 7f9a1be00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c0af040 0x7f9a1c144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.202+0000 7f9a1be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c145520 0x7f9a1c158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.202+0000 7f9a1be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a1c142b10 0x7f9a1c0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.202+0000 7f9a1be00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c06e590 con 0x7f9a1c145520 2024-03-08T06:03:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.202+0000 7f9a1be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c06e410 con 0x7f9a1c142b10 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.202+0000 7f9a1be00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c06e710 con 0x7f9a1c0af040 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.202+0000 7f9a13e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c145520 0x7f9a1c158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.202+0000 7f9a13e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c145520 0x7f9a1c158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41234/0 (socket says 172.21.15.40:41234) 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.202+0000 7f9a13e00640 1 -- 172.21.15.40:0/2958047563 learned_addr learned my addr 172.21.15.40:0/2958047563 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a1b400640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a1c142b10 0x7f9a1c0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a13e00640 1 -- 172.21.15.40:0/2958047563 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c0af040 msgr2=0x7f9a1c144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a13e00640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c0af040 0x7f9a1c144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a13e00640 1 -- 172.21.15.40:0/2958047563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a1c142b10 msgr2=0x7f9a1c0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a13e00640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a1c142b10 0x7f9a1c0b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a13e00640 1 -- 172.21.15.40:0/2958047563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c159ac0 con 0x7f9a1c145520 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a13e00640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c145520 0x7f9a1c158c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a0402b910 tx=0x7f9a040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a13400640 1 -- 172.21.15.40:0/2958047563 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04040070 con 0x7f9a1c145520 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a13400640 1 -- 172.21.15.40:0/2958047563 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0403a910 con 0x7f9a1c145520 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a1be00640 1 -- 172.21.15.40:0/2958047563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c143010 con 0x7f9a1c145520 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a13400640 1 -- 172.21.15.40:0/2958047563 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04035040 con 0x7f9a1c145520 2024-03-08T06:03:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.203+0000 7f9a1be00640 1 -- 172.21.15.40:0/2958047563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c1435d0 con 0x7f9a1c145520 2024-03-08T06:03:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.204+0000 7f9a1be00640 1 -- 172.21.15.40:0/2958047563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c075100 con 0x7f9a1c145520 2024-03-08T06:03:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.205+0000 7f9a13400640 1 -- 172.21.15.40:0/2958047563 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f9a0404b060 con 0x7f9a1c145520 2024-03-08T06:03:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.206+0000 7f9a13400640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f99f806b470 0x7f99f806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.206+0000 7f9a13400640 1 -- 172.21.15.40:0/2958047563 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f9a040ab420 con 0x7f9a1c145520 2024-03-08T06:03:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.206+0000 7f9a1aa00640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f99f806b470 0x7f99f806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.207+0000 7f9a1aa00640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f99f806b470 0x7f99f806d930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c006a90 tx=0x7f9a0c006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:55.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.211+0000 7f9a13400640 1 -- 172.21.15.40:0/2958047563 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a0407a7b0 con 0x7f9a1c145520 2024-03-08T06:03:55.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.438+0000 7f9a1be00640 1 -- 172.21.15.40:0/2958047563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a1c0747e0 con 0x7f9a1c145520 2024-03-08T06:03:55.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.439+0000 7f9a13400640 1 -- 172.21.15.40:0/2958047563 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f9a0407a0b0 con 0x7f9a1c145520 2024-03-08T06:03:55.439 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:55.439 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3709744112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3709744112},{"type":"v1","addr":"172.21.15.40:6835","nonce":3709744112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-08T06:03:55.439 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 2 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 -- 172.21.15.40:0/2958047563 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f99f806b470 msgr2=0x7f99f806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f99f806b470 0x7f99f806d930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c006a90 tx=0x7f9a0c006a20 comp rx=0 tx=0).stop 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 -- 172.21.15.40:0/2958047563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c145520 msgr2=0x7f9a1c158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c145520 0x7f9a1c158c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a0402b910 tx=0x7f9a040046c0 comp rx=0 tx=0).stop 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 -- 172.21.15.40:0/2958047563 shutdown_connections 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f99f806b470 0x7f99f806d930 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a1c142b10 0x7f9a1c0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c145520 0x7f9a1c158c80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 --2- 172.21.15.40:0/2958047563 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c0af040 0x7f9a1c144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 -- 172.21.15.40:0/2958047563 >> 172.21.15.40:0/2958047563 conn(0x7f9a1c0b1850 msgr2=0x7f9a1c06ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 -- 172.21.15.40:0/2958047563 shutdown_connections 2024-03-08T06:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:55.441+0000 7f9a1be00640 1 -- 172.21.15.40:0/2958047563 wait complete. 2024-03-08T06:03:57.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:03:57.529 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:03:57.535 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:03:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.680+0000 7f9e41a00640 1 Processor -- start 2024-03-08T06:03:57.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.681+0000 7f9e41a00640 1 -- start start 2024-03-08T06:03:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.682+0000 7f9e41a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 0x7f9e3c0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.682+0000 7f9e41a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9e3c15e580 0x7f9e3c06e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.682+0000 7f9e41a00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c15eaf0 con 0x7f9e3c06ec50 2024-03-08T06:03:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.682+0000 7f9e41a00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c06f0b0 con 0x7f9e3c15e580 2024-03-08T06:03:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.682+0000 7f9e41a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0b3350 con 0x7f9e3c144030 2024-03-08T06:03:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.683+0000 7f9e32000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9e3c15e580 0x7f9e3c06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.683+0000 7f9e3b400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9e3c06ec50 0x7f9e3c071840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41958/0 (socket says 172.21.15.40:41958) 2024-03-08T06:03:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.683+0000 7f9e3b400640 1 -- 172.21.15.40:0/1748591388 learned_addr learned my addr 172.21.15.40:0/1748591388 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:03:57.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.683+0000 7f9e3aa00640 1 --2- 172.21.15.40:0/1748591388 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 0x7f9e3c0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.683+0000 7f9e3a000640 1 -- 172.21.15.40:0/1748591388 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978329905 0 0) 0x7f9e3c15eaf0 con 0x7f9e3c06ec50 2024-03-08T06:03:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.683+0000 7f9e3a000640 1 -- 172.21.15.40:0/1748591388 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e20005730 con 0x7f9e3c06ec50 2024-03-08T06:03:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.683+0000 7f9e32000640 1 -- 172.21.15.40:0/1748591388 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 msgr2=0x7f9e3c0cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.683+0000 7f9e32000640 1 --2- 172.21.15.40:0/1748591388 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 0x7f9e3c0cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.683+0000 7f9e32000640 1 -- 172.21.15.40:0/1748591388 >> v1:172.21.15.40:6789/0 conn(0x7f9e3c06ec50 legacy=0x7f9e3c071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.683+0000 7f9e32000640 1 -- 172.21.15.40:0/1748591388 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c15edd0 con 0x7f9e3c15e580 2024-03-08T06:03:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.683+0000 7f9e3aa00640 1 --2- 172.21.15.40:0/1748591388 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 0x7f9e3c0cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:03:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.684+0000 7f9e32000640 1 --2- 172.21.15.40:0/1748591388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e3c15e580 0x7f9e3c06e710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9e240097a0 tx=0x7f9e2402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909ea1787a7db638 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.685+0000 7f9e3a000640 1 -- 172.21.15.40:0/1748591388 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24035020 con 0x7f9e3c15e580 2024-03-08T06:03:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.685+0000 7f9e3a000640 1 -- 172.21.15.40:0/1748591388 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e24031830 con 0x7f9e3c15e580 2024-03-08T06:03:57.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.685+0000 7f9e3a000640 1 -- 172.21.15.40:0/1748591388 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24031c00 con 0x7f9e3c15e580 2024-03-08T06:03:57.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.685+0000 7f9e41a00640 1 -- 172.21.15.40:0/1748591388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e3c15e580 msgr2=0x7f9e3c06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:57.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.685+0000 7f9e41a00640 1 --2- 172.21.15.40:0/1748591388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e3c15e580 0x7f9e3c06e710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9e240097a0 tx=0x7f9e2402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:03:57.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.686+0000 7f9e41a00640 1 -- 172.21.15.40:0/1748591388 shutdown_connections 2024-03-08T06:03:57.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.686+0000 7f9e41a00640 1 --2- 172.21.15.40:0/1748591388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e3c15e580 0x7f9e3c06e710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:57.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.686+0000 7f9e41a00640 1 --2- 172.21.15.40:0/1748591388 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 0x7f9e3c0cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:57.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.686+0000 7f9e41a00640 1 -- 172.21.15.40:0/1748591388 >> 172.21.15.40:0/1748591388 conn(0x7f9e3c06ae30 msgr2=0x7f9e3c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:57.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.686+0000 7f9e41a00640 1 -- 172.21.15.40:0/1748591388 shutdown_connections 2024-03-08T06:03:57.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.686+0000 7f9e41a00640 1 -- 172.21.15.40:0/1748591388 wait complete. 2024-03-08T06:03:57.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.687+0000 7f9e41a00640 1 Processor -- start 2024-03-08T06:03:57.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.688+0000 7f9e41a00640 1 -- start start 2024-03-08T06:03:57.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.688+0000 7f9e41a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e3c06ec50 0x7f9e3c131250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:57.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.688+0000 7f9e41a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 0x7f9e3c15a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:57.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.688+0000 7f9e41a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e3c15e580 0x7f9e3c141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:57.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.688+0000 7f9e41a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0cf5c0 con 0x7f9e3c15e580 2024-03-08T06:03:57.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.688+0000 7f9e41a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c15f390 con 0x7f9e3c06ec50 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.688+0000 7f9e41a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0cf740 con 0x7f9e3c144030 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.689+0000 7f9e3b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e3c15e580 0x7f9e3c141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.689+0000 7f9e3b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e3c15e580 0x7f9e3c141c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59708/0 (socket says 172.21.15.40:59708) 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.689+0000 7f9e3b400640 1 -- 172.21.15.40:0/1826186579 learned_addr learned my addr 172.21.15.40:0/1826186579 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.689+0000 7f9e32000640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 0x7f9e3c15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.689+0000 7f9e3aa00640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e3c06ec50 0x7f9e3c131250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.689+0000 7f9e3b400640 1 -- 172.21.15.40:0/1826186579 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 msgr2=0x7f9e3c15a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.689+0000 7f9e3b400640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 0x7f9e3c15a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.689+0000 7f9e3b400640 1 -- 172.21.15.40:0/1826186579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e3c06ec50 msgr2=0x7f9e3c131250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.689+0000 7f9e3b400640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e3c06ec50 0x7f9e3c131250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.689+0000 7f9e3b400640 1 -- 172.21.15.40:0/1826186579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c131790 con 0x7f9e3c15e580 2024-03-08T06:03:57.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.689+0000 7f9e32000640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 0x7f9e3c15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:03:57.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.690+0000 7f9e3b400640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e3c15e580 0x7f9e3c141c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400cb40 tx=0x7f9e3400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:57.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.690+0000 7f9e33400640 1 -- 172.21.15.40:0/1826186579 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34017070 con 0x7f9e3c15e580 2024-03-08T06:03:57.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.690+0000 7f9e33400640 1 -- 172.21.15.40:0/1826186579 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3400aca0 con 0x7f9e3c15e580 2024-03-08T06:03:57.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.690+0000 7f9e41a00640 1 -- 172.21.15.40:0/1826186579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c15a970 con 0x7f9e3c15e580 2024-03-08T06:03:57.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.690+0000 7f9e33400640 1 -- 172.21.15.40:0/1826186579 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34012620 con 0x7f9e3c15e580 2024-03-08T06:03:57.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.690+0000 7f9e41a00640 1 -- 172.21.15.40:0/1826186579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c1e4ae0 con 0x7f9e3c15e580 2024-03-08T06:03:57.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.691+0000 7f9e41a00640 1 -- 172.21.15.40:0/1826186579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c067cb0 con 0x7f9e3c15e580 2024-03-08T06:03:57.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.692+0000 7f9e33400640 1 -- 172.21.15.40:0/1826186579 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f9e3401a070 con 0x7f9e3c15e580 2024-03-08T06:03:57.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.693+0000 7f9e33400640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9e1806b440 0x7f9e1806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:57.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.693+0000 7f9e33400640 1 -- 172.21.15.40:0/1826186579 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9e34089bf0 con 0x7f9e3c15e580 2024-03-08T06:03:57.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.693+0000 7f9e3aa00640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9e1806b440 0x7f9e1806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:57.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.694+0000 7f9e3aa00640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9e1806b440 0x7f9e1806d900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c006fd0 tx=0x7f9e2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.703+0000 7f9e33400640 1 -- 172.21.15.40:0/1826186579 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e3c131790 con 0x7f9e3c15e580 2024-03-08T06:03:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.950+0000 7f9e41a00640 1 -- 172.21.15.40:0/1826186579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e3c0aeec0 con 0x7f9e3c15e580 2024-03-08T06:03:57.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.951+0000 7f9e33400640 1 -- 172.21.15.40:0/1826186579 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f9e34058ea0 con 0x7f9e3c15e580 2024-03-08T06:03:57.952 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:57.952 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3266388900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3266388900},{"type":"v1","addr":"172.21.15.40:6837","nonce":3266388900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2471668569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2471668569},{"type":"v1","addr":"172.21.15.53:6834","nonce":2471668569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3262228631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3262228631},{"type":"v1","addr":"172.21.15.53:6835","nonce":3262228631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3709744112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3709744112},{"type":"v1","addr":"172.21.15.40:6835","nonce":3709744112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-08T06:03:57.953 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 3 2024-03-08T06:03:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.953+0000 7f9e41a00640 1 -- 172.21.15.40:0/1826186579 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9e1806b440 msgr2=0x7f9e1806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:57.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.953+0000 7f9e41a00640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9e1806b440 0x7f9e1806d900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c006fd0 tx=0x7f9e2c008040 comp rx=0 tx=0).stop 2024-03-08T06:03:57.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.953+0000 7f9e41a00640 1 -- 172.21.15.40:0/1826186579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e3c15e580 msgr2=0x7f9e3c141c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:57.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.953+0000 7f9e41a00640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e3c15e580 0x7f9e3c141c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400cb40 tx=0x7f9e3400a060 comp rx=0 tx=0).stop 2024-03-08T06:03:57.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.954+0000 7f9e41a00640 1 -- 172.21.15.40:0/1826186579 shutdown_connections 2024-03-08T06:03:57.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.954+0000 7f9e41a00640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9e1806b440 0x7f9e1806d900 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:57.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.954+0000 7f9e41a00640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e3c15e580 0x7f9e3c141c20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:57.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.954+0000 7f9e41a00640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9e3c144030 0x7f9e3c15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:57.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.954+0000 7f9e41a00640 1 --2- 172.21.15.40:0/1826186579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e3c06ec50 0x7f9e3c131250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:57.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.954+0000 7f9e41a00640 1 -- 172.21.15.40:0/1826186579 >> 172.21.15.40:0/1826186579 conn(0x7f9e3c06ae30 msgr2=0x7f9e3c0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:57.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.954+0000 7f9e41a00640 1 -- 172.21.15.40:0/1826186579 shutdown_connections 2024-03-08T06:03:57.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:57.954+0000 7f9e41a00640 1 -- 172.21.15.40:0/1826186579 wait complete. 2024-03-08T06:03:57.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:03:58.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.177+0000 7f0e81600640 1 Processor -- start 2024-03-08T06:03:58.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.178+0000 7f0e81600640 1 -- start start 2024-03-08T06:03:58.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.179+0000 7f0e81600640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e7c1613b0 0x7f0e7c0d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.179+0000 7f0e81600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0e7c0b9500 0x7f0e7c0d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.179+0000 7f0e81600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c0d25b0 con 0x7f0e7c06c3e0 2024-03-08T06:03:58.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.179+0000 7f0e81600640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7c0d2890 con 0x7f0e7c1613b0 2024-03-08T06:03:58.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.179+0000 7f0e81600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0d2b70 con 0x7f0e7c0b9500 2024-03-08T06:03:58.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.179+0000 7f0e7aa00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e7c1613b0 0x7f0e7c0d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.179+0000 7f0e7aa00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e7c1613b0 0x7f0e7c0d3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59710/0 (socket says 172.21.15.40:59710) 2024-03-08T06:03:58.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.179+0000 7f0e7aa00640 1 -- 172.21.15.40:0/1489179059 learned_addr learned my addr 172.21.15.40:0/1489179059 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:58.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.179+0000 7f0e73e00640 1 --2- 172.21.15.40:0/1489179059 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0e7c0b9500 0x7f0e7c0d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.180+0000 7f0e7aa00640 1 -- 172.21.15.40:0/1489179059 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0e7c0b9500 msgr2=0x7f0e7c0d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.180+0000 7f0e7aa00640 1 --2- 172.21.15.40:0/1489179059 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0e7c0b9500 0x7f0e7c0d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.180+0000 7f0e7aa00640 1 -- 172.21.15.40:0/1489179059 >> v1:172.21.15.53:6789/0 conn(0x7f0e7c06c3e0 legacy=0x7f0e7c0b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:58.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.180+0000 7f0e7aa00640 1 -- 172.21.15.40:0/1489179059 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c0d2e50 con 0x7f0e7c1613b0 2024-03-08T06:03:58.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.180+0000 7f0e73e00640 1 --2- 172.21.15.40:0/1489179059 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0e7c0b9500 0x7f0e7c0d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:03:58.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.181+0000 7f0e7aa00640 1 --2- 172.21.15.40:0/1489179059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7c1613b0 0x7f0e7c0d3650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009870 tx=0x7f0e6c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1be40ee3c3b93f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:58.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.181+0000 7f0e7a000640 1 -- 172.21.15.40:0/1489179059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c034070 con 0x7f0e7c1613b0 2024-03-08T06:03:58.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.181+0000 7f0e7a000640 1 -- 172.21.15.40:0/1489179059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c030840 con 0x7f0e7c1613b0 2024-03-08T06:03:58.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.181+0000 7f0e7a000640 1 -- 172.21.15.40:0/1489179059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c030c10 con 0x7f0e7c1613b0 2024-03-08T06:03:58.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.181+0000 7f0e81600640 1 -- 172.21.15.40:0/1489179059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7c1613b0 msgr2=0x7f0e7c0d3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.181+0000 7f0e81600640 1 --2- 172.21.15.40:0/1489179059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7c1613b0 0x7f0e7c0d3650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009870 tx=0x7f0e6c02b980 comp rx=0 tx=0).stop 2024-03-08T06:03:58.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.182+0000 7f0e81600640 1 -- 172.21.15.40:0/1489179059 shutdown_connections 2024-03-08T06:03:58.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.182+0000 7f0e81600640 1 --2- 172.21.15.40:0/1489179059 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0e7c0b9500 0x7f0e7c0d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.182+0000 7f0e81600640 1 --2- 172.21.15.40:0/1489179059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7c1613b0 0x7f0e7c0d3650 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.182+0000 7f0e81600640 1 -- 172.21.15.40:0/1489179059 >> 172.21.15.40:0/1489179059 conn(0x7f0e7c0729b0 msgr2=0x7f0e7c0d3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:58.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.182+0000 7f0e81600640 1 -- 172.21.15.40:0/1489179059 shutdown_connections 2024-03-08T06:03:58.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.182+0000 7f0e81600640 1 -- 172.21.15.40:0/1489179059 wait complete. 2024-03-08T06:03:58.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.183+0000 7f0e81600640 1 Processor -- start 2024-03-08T06:03:58.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.184+0000 7f0e81600640 1 -- start start 2024-03-08T06:03:58.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.184+0000 7f0e81600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0e7c06c3e0 0x7f0e7c15c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.184+0000 7f0e81600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e7c0b9500 0x7f0e7c0ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.184+0000 7f0e81600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7c1613b0 0x7f0e7c1f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.184+0000 7f0e81600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c161790 con 0x7f0e7c1613b0 2024-03-08T06:03:58.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e81600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0d22a0 con 0x7f0e7c0b9500 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e81600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c06c830 con 0x7f0e7c06c3e0 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e7b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7c1613b0 0x7f0e7c1f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e7b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7c1613b0 0x7f0e7c1f6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59722/0 (socket says 172.21.15.40:59722) 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e7b400640 1 -- 172.21.15.40:0/843003599 learned_addr learned my addr 172.21.15.40:0/843003599 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e73e00640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e7c0b9500 0x7f0e7c0ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e7aa00640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0e7c06c3e0 0x7f0e7c15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e7b400640 1 -- 172.21.15.40:0/843003599 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0e7c06c3e0 msgr2=0x7f0e7c15c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e7b400640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0e7c06c3e0 0x7f0e7c15c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e7b400640 1 -- 172.21.15.40:0/843003599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e7c0b9500 msgr2=0x7f0e7c0ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e7b400640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e7c0b9500 0x7f0e7c0ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e7b400640 1 -- 172.21.15.40:0/843003599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c0b47f0 con 0x7f0e7c1613b0 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.185+0000 7f0e73e00640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e7c0b9500 0x7f0e7c0ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:03:58.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.186+0000 7f0e7b400640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7c1613b0 0x7f0e7c1f6bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400c7b0 tx=0x7f0e7400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:58.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.186+0000 7f0e72a00640 1 -- 172.21.15.40:0/843003599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74004b60 con 0x7f0e7c1613b0 2024-03-08T06:03:58.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.186+0000 7f0e72a00640 1 -- 172.21.15.40:0/843003599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e74004d00 con 0x7f0e7c1613b0 2024-03-08T06:03:58.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.186+0000 7f0e72a00640 1 -- 172.21.15.40:0/843003599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400aa90 con 0x7f0e7c1613b0 2024-03-08T06:03:58.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.186+0000 7f0e81600640 1 -- 172.21.15.40:0/843003599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c0b4ad0 con 0x7f0e7c1613b0 2024-03-08T06:03:58.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.186+0000 7f0e81600640 1 -- 172.21.15.40:0/843003599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c0b5090 con 0x7f0e7c1613b0 2024-03-08T06:03:58.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.187+0000 7f0e81600640 1 -- 172.21.15.40:0/843003599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c0683c0 con 0x7f0e7c1613b0 2024-03-08T06:03:58.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.189+0000 7f0e72a00640 1 -- 172.21.15.40:0/843003599 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f0e7400ac30 con 0x7f0e7c1613b0 2024-03-08T06:03:58.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.190+0000 7f0e72a00640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0e5806b440 0x7f0e5806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.190+0000 7f0e72a00640 1 -- 172.21.15.40:0/843003599 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0e7408a060 con 0x7f0e7c1613b0 2024-03-08T06:03:58.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.190+0000 7f0e7aa00640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0e5806b440 0x7f0e5806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.191+0000 7f0e7aa00640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0e5806b440 0x7f0e5806d900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c036000 tx=0x7f0e6c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:58.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.195+0000 7f0e72a00640 1 -- 172.21.15.40:0/843003599 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e74020070 con 0x7f0e7c1613b0 2024-03-08T06:03:58.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.397+0000 7f0e81600640 1 -- 172.21.15.40:0/843003599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e7c15f230 con 0x7f0e7c1613b0 2024-03-08T06:03:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.399+0000 7f0e72a00640 1 -- 172.21.15.40:0/843003599 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12800 (secure 0 0 0) 0x7f0e74059310 con 0x7f0e7c1613b0 2024-03-08T06:03:58.399 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:58.399 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:03:58.236114+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.401+0000 7f0e81600640 1 -- 172.21.15.40:0/843003599 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0e5806b440 msgr2=0x7f0e5806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.401+0000 7f0e81600640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0e5806b440 0x7f0e5806d900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c036000 tx=0x7f0e6c03b040 comp rx=0 tx=0).stop 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.401+0000 7f0e81600640 1 -- 172.21.15.40:0/843003599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7c1613b0 msgr2=0x7f0e7c1f6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.401+0000 7f0e81600640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7c1613b0 0x7f0e7c1f6bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400c7b0 tx=0x7f0e7400cc70 comp rx=0 tx=0).stop 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.401+0000 7f0e81600640 1 -- 172.21.15.40:0/843003599 shutdown_connections 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.401+0000 7f0e81600640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0e5806b440 0x7f0e5806d900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.401+0000 7f0e81600640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e7c1613b0 0x7f0e7c1f6bb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.401+0000 7f0e81600640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e7c0b9500 0x7f0e7c0ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.401+0000 7f0e81600640 1 --2- 172.21.15.40:0/843003599 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0e7c06c3e0 0x7f0e7c15c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.401+0000 7f0e81600640 1 -- 172.21.15.40:0/843003599 >> 172.21.15.40:0/843003599 conn(0x7f0e7c0729b0 msgr2=0x7f0e7c0c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.401+0000 7f0e81600640 1 -- 172.21.15.40:0/843003599 shutdown_connections 2024-03-08T06:03:58.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.402+0000 7f0e81600640 1 -- 172.21.15.40:0/843003599 wait complete. 2024-03-08T06:03:58.422 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:03:58.422 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:03:58.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.615+0000 7f8f92200640 1 Processor -- start 2024-03-08T06:03:58.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.615+0000 7f8f92200640 1 -- start start 2024-03-08T06:03:58.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.617+0000 7f8f92200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f8c06b360 0x7f8f8c0cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.617+0000 7f8f92200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8c0d0420 con 0x7f8f8c0cd5a0 2024-03-08T06:03:58.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.617+0000 7f8f92200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8c0d0700 con 0x7f8f8c15e070 2024-03-08T06:03:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.617+0000 7f8f92200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0d01f0 con 0x7f8f8c06b360 2024-03-08T06:03:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.617+0000 7f8f83e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8f8c0cd5a0 0x7f8f8c06ac50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41968/0 (socket says 172.21.15.40:41968) 2024-03-08T06:03:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.617+0000 7f8f83e00640 1 -- 172.21.15.40:0/312739524 learned_addr learned my addr 172.21.15.40:0/312739524 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:03:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.617+0000 7f8f8be00640 1 --2- 172.21.15.40:0/312739524 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f8c06b360 0x7f8f8c0cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.617+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237250556 0 0) 0x7f8f8c0d0420 con 0x7f8f8c0cd5a0 2024-03-08T06:03:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f70003520 con 0x7f8f8c0cd5a0 2024-03-08T06:03:58.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2297582199 0 0) 0x7f8f70003520 con 0x7f8f8c0cd5a0 2024-03-08T06:03:58.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f8c06b360 msgr2=0x7f8f8c0cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:58.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 --2- 172.21.15.40:0/312739524 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f8c06b360 0x7f8f8c0cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 >> v1:172.21.15.53:6789/0 conn(0x7f8f8c15e070 legacy=0x7f8f8c06e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c0cb0e0 con 0x7f8f8c0cd5a0 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f8f74001670 con 0x7f8f8c0cd5a0 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 >> v1:172.21.15.40:6789/0 conn(0x7f8f8c0cd5a0 legacy=0x7f8f8c06ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 --2- 172.21.15.40:0/312739524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f700039e0 0x7f8f70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 --2- 172.21.15.40:0/312739524 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f8c06b360 0x7f8f70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 --2- 172.21.15.40:0/312739524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f70004d40 0x7f8f700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7000ba70 con 0x7f8f70004d40 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7000bc20 con 0x7f8f700039e0 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7000be20 con 0x7f8f8c06b360 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.618+0000 7f8f8be00640 1 --2- 172.21.15.40:0/312739524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f70004d40 0x7f8f700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.619+0000 7f8f8b400640 1 --2- 172.21.15.40:0/312739524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f700039e0 0x7f8f70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.619+0000 7f8f8be00640 1 -- 172.21.15.40:0/312739524 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f8c06b360 msgr2=0x7f8f70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.619+0000 7f8f8be00640 1 --2- 172.21.15.40:0/312739524 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f8c06b360 0x7f8f70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.619+0000 7f8f8be00640 1 -- 172.21.15.40:0/312739524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f700039e0 msgr2=0x7f8f70003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.619+0000 7f8f8be00640 1 --2- 172.21.15.40:0/312739524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f700039e0 0x7f8f70003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.619+0000 7f8f8be00640 1 -- 172.21.15.40:0/312739524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f70003520 con 0x7f8f70004d40 2024-03-08T06:03:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.620+0000 7f8f8be00640 1 --2- 172.21.15.40:0/312739524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f70004d40 0x7f8f700051c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009f90 tx=0x7f8f70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62dd7ab5e9d2069e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.620+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f84039070 con 0x7f8f70004d40 2024-03-08T06:03:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.620+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8402cc20 con 0x7f8f70004d40 2024-03-08T06:03:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.620+0000 7f8f8aa00640 1 -- 172.21.15.40:0/312739524 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f84009c10 con 0x7f8f70004d40 2024-03-08T06:03:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.621+0000 7f8f92200640 1 -- 172.21.15.40:0/312739524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f70004d40 msgr2=0x7f8f700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.621+0000 7f8f92200640 1 --2- 172.21.15.40:0/312739524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f70004d40 0x7f8f700051c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009f90 tx=0x7f8f70005830 comp rx=0 tx=0).stop 2024-03-08T06:03:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.621+0000 7f8f8be00640 1 -- 172.21.15.40:0/312739524 reap_dead start 2024-03-08T06:03:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.621+0000 7f8f92200640 1 -- 172.21.15.40:0/312739524 shutdown_connections 2024-03-08T06:03:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.621+0000 7f8f92200640 1 -- 172.21.15.40:0/312739524 >> 172.21.15.40:0/312739524 conn(0x7f8f8c099a70 msgr2=0x7f8f8c099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.621+0000 7f8f92200640 1 -- 172.21.15.40:0/312739524 shutdown_connections 2024-03-08T06:03:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.621+0000 7f8f92200640 1 -- 172.21.15.40:0/312739524 wait complete. 2024-03-08T06:03:58.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.623+0000 7f8f92200640 1 Processor -- start 2024-03-08T06:03:58.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.623+0000 7f8f92200640 1 -- start start 2024-03-08T06:03:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.624+0000 7f8f92200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f70004d40 0x7f8f8c14b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.624+0000 7f8f92200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c1f6ed0 0x7f8f8c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.624+0000 7f8f92200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c0c7300 0x7f8f8c0cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.624+0000 7f8f92200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c15e450 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.624+0000 7f8f92200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0cb6a0 con 0x7f8f8c0c7300 2024-03-08T06:03:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.624+0000 7f8f92200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0cfcb0 con 0x7f8f70004d40 2024-03-08T06:03:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.625+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c1f6ed0 0x7f8f8c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.625+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c1f6ed0 0x7f8f8c0c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59744/0 (socket says 172.21.15.40:59744) 2024-03-08T06:03:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.625+0000 7f8f83e00640 1 -- 172.21.15.40:0/36753582 learned_addr learned my addr 172.21.15.40:0/36753582 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.625+0000 7f8f8b400640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f70004d40 0x7f8f8c14b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.625+0000 7f8f8be00640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c0c7300 0x7f8f8c0cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.625+0000 7f8f83e00640 1 -- 172.21.15.40:0/36753582 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f70004d40 msgr2=0x7f8f8c14b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.625+0000 7f8f83e00640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f70004d40 0x7f8f8c14b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.625+0000 7f8f83e00640 1 -- 172.21.15.40:0/36753582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c0c7300 msgr2=0x7f8f8c0cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.625+0000 7f8f83e00640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c0c7300 0x7f8f8c0cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.625+0000 7f8f83e00640 1 -- 172.21.15.40:0/36753582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c14bd40 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.625+0000 7f8f83e00640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c1f6ed0 0x7f8f8c0c6dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c700 tx=0x7f8f7400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.626+0000 7f8f83400640 1 -- 172.21.15.40:0/36753582 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74018070 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.626+0000 7f8f83400640 1 -- 172.21.15.40:0/36753582 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f740074e0 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.626+0000 7f8f83400640 1 -- 172.21.15.40:0/36753582 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74004a20 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.626+0000 7f8f92200640 1 -- 172.21.15.40:0/36753582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c1f7400 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.626+0000 7f8f92200640 1 -- 172.21.15.40:0/36753582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c1f79c0 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.627+0000 7f8f92200640 1 -- 172.21.15.40:0/36753582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c0683c0 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.629+0000 7f8f83400640 1 -- 172.21.15.40:0/36753582 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f8f74004bc0 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.630+0000 7f8f83400640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8f6806b440 0x7f8f6806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:58.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.630+0000 7f8f83400640 1 -- 172.21.15.40:0/36753582 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8f740893e0 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.630+0000 7f8f8b400640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8f6806b440 0x7f8f6806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:58.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.631+0000 7f8f8b400640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8f6806b440 0x7f8f6806d900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0055b0 tx=0x7f8f7c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.635+0000 7f8f83400640 1 -- 172.21.15.40:0/36753582 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f8c14bd40 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.836+0000 7f8f92200640 1 -- 172.21.15.40:0/36753582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f8c0ada60 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.836+0000 7f8f83400640 1 -- 172.21.15.40:0/36753582 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f740585f0 con 0x7f8f8c1f6ed0 2024-03-08T06:03:58.836 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:03:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.839+0000 7f8f92200640 1 -- 172.21.15.40:0/36753582 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8f6806b440 msgr2=0x7f8f6806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.839+0000 7f8f92200640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8f6806b440 0x7f8f6806d900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0055b0 tx=0x7f8f7c009040 comp rx=0 tx=0).stop 2024-03-08T06:03:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.839+0000 7f8f92200640 1 -- 172.21.15.40:0/36753582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c1f6ed0 msgr2=0x7f8f8c0c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.839+0000 7f8f92200640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c1f6ed0 0x7f8f8c0c6dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c700 tx=0x7f8f7400cbc0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.839+0000 7f8f92200640 1 -- 172.21.15.40:0/36753582 shutdown_connections 2024-03-08T06:03:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.839+0000 7f8f92200640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8f6806b440 0x7f8f6806d900 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.839+0000 7f8f92200640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c0c7300 0x7f8f8c0cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.839+0000 7f8f92200640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f8c1f6ed0 0x7f8f8c0c6dc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.839+0000 7f8f92200640 1 --2- 172.21.15.40:0/36753582 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f70004d40 0x7f8f8c14b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.839+0000 7f8f92200640 1 -- 172.21.15.40:0/36753582 >> 172.21.15.40:0/36753582 conn(0x7f8f8c099a70 msgr2=0x7f8f8c0ca3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.840+0000 7f8f92200640 1 -- 172.21.15.40:0/36753582 shutdown_connections 2024-03-08T06:03:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:58.840+0000 7f8f92200640 1 -- 172.21.15.40:0/36753582 wait complete. 2024-03-08T06:03:58.859 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-08T06:03:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.086+0000 7fa3db200640 1 Processor -- start 2024-03-08T06:03:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.086+0000 7fa3db200640 1 -- start start 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.088+0000 7fa3db200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa3d415df30 0x7fa3d4161a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.088+0000 7fa3db200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3d40b9790 0x7fa3d40cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.088+0000 7fa3db200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3d40cefc0 0x7fa3d40b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.088+0000 7fa3db200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d40cf430 con 0x7fa3d40cefc0 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.088+0000 7fa3db200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d41629a0 con 0x7fa3d40b9790 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.088+0000 7fa3db200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d4162100 con 0x7fa3d415df30 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.088+0000 7fa3d9200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3d40cefc0 0x7fa3d40b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.088+0000 7fa3d9200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3d40cefc0 0x7fa3d40b5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59748/0 (socket says 172.21.15.40:59748) 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.088+0000 7fa3d9200640 1 -- 172.21.15.40:0/2198288917 learned_addr learned my addr 172.21.15.40:0/2198288917 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.088+0000 7fa3d3400640 1 --2- 172.21.15.40:0/2198288917 >> v2:172.21.15.53:3300/0 conn(0x7fa3d40b9790 0x7fa3d40cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.088+0000 7fa3d3e00640 1 --2- 172.21.15.40:0/2198288917 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa3d415df30 0x7fa3d4161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.089+0000 7fa3d9200640 1 -- 172.21.15.40:0/2198288917 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa3d415df30 msgr2=0x7fa3d4161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:59.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.089+0000 7fa3d9200640 1 --2- 172.21.15.40:0/2198288917 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa3d415df30 0x7fa3d4161a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.089+0000 7fa3d9200640 1 -- 172.21.15.40:0/2198288917 >> v2:172.21.15.53:3300/0 conn(0x7fa3d40b9790 msgr2=0x7fa3d40cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:59.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.089+0000 7fa3d9200640 1 --2- 172.21.15.40:0/2198288917 >> v2:172.21.15.53:3300/0 conn(0x7fa3d40b9790 0x7fa3d40cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.089+0000 7fa3d9200640 1 -- 172.21.15.40:0/2198288917 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d41623e0 con 0x7fa3d40cefc0 2024-03-08T06:03:59.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.090+0000 7fa3d9200640 1 --2- 172.21.15.40:0/2198288917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d40cefc0 0x7fa3d40b5730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40b67b0 tx=0x7fa3cc02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a675467c2105d45 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:59.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.090+0000 7fa3d2a00640 1 -- 172.21.15.40:0/2198288917 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc02c950 con 0x7fa3d40cefc0 2024-03-08T06:03:59.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.090+0000 7fa3d2a00640 1 -- 172.21.15.40:0/2198288917 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc02caf0 con 0x7fa3d40cefc0 2024-03-08T06:03:59.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.090+0000 7fa3d2a00640 1 -- 172.21.15.40:0/2198288917 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0343f0 con 0x7fa3d40cefc0 2024-03-08T06:03:59.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.091+0000 7fa3db200640 1 -- 172.21.15.40:0/2198288917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d40cefc0 msgr2=0x7fa3d40b5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:59.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.091+0000 7fa3db200640 1 --2- 172.21.15.40:0/2198288917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d40cefc0 0x7fa3d40b5730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40b67b0 tx=0x7fa3cc02bdb0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.091+0000 7fa3db200640 1 -- 172.21.15.40:0/2198288917 shutdown_connections 2024-03-08T06:03:59.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.091+0000 7fa3db200640 1 --2- 172.21.15.40:0/2198288917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d40cefc0 0x7fa3d40b5730 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.091+0000 7fa3db200640 1 --2- 172.21.15.40:0/2198288917 >> v2:172.21.15.53:3300/0 conn(0x7fa3d40b9790 0x7fa3d40cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.091+0000 7fa3db200640 1 --2- 172.21.15.40:0/2198288917 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa3d415df30 0x7fa3d4161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.091+0000 7fa3db200640 1 -- 172.21.15.40:0/2198288917 >> 172.21.15.40:0/2198288917 conn(0x7fa3d406c600 msgr2=0x7fa3d406ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:59.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.091+0000 7fa3db200640 1 -- 172.21.15.40:0/2198288917 shutdown_connections 2024-03-08T06:03:59.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.091+0000 7fa3db200640 1 -- 172.21.15.40:0/2198288917 wait complete. 2024-03-08T06:03:59.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.093+0000 7fa3db200640 1 Processor -- start 2024-03-08T06:03:59.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.093+0000 7fa3db200640 1 -- start start 2024-03-08T06:03:59.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.094+0000 7fa3db200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa3d40b9790 0x7fa3d40c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.094+0000 7fa3db200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d40cefc0 0x7fa3d4151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.094+0000 7fa3db200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d415df30 0x7fa3d4151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.094+0000 7fa3db200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40b9bf0 con 0x7fa3d415df30 2024-03-08T06:03:59.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.094+0000 7fa3db200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40d2220 con 0x7fa3d40cefc0 2024-03-08T06:03:59.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.094+0000 7fa3db200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d415e310 con 0x7fa3d40b9790 2024-03-08T06:03:59.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.095+0000 7fa3d9200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d415df30 0x7fa3d4151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.095+0000 7fa3d9200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d415df30 0x7fa3d4151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59750/0 (socket says 172.21.15.40:59750) 2024-03-08T06:03:59.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.095+0000 7fa3d9200640 1 -- 172.21.15.40:0/2463756317 learned_addr learned my addr 172.21.15.40:0/2463756317 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:59.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.095+0000 7fa3d3400640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d40cefc0 0x7fa3d4151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.095+0000 7fa3d3e00640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa3d40b9790 0x7fa3d40c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.095+0000 7fa3d9200640 1 -- 172.21.15.40:0/2463756317 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa3d40b9790 msgr2=0x7fa3d40c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:59.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.095+0000 7fa3d9200640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa3d40b9790 0x7fa3d40c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.095+0000 7fa3d9200640 1 -- 172.21.15.40:0/2463756317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d40cefc0 msgr2=0x7fa3d4151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:59.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.095+0000 7fa3d9200640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d40cefc0 0x7fa3d4151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.095+0000 7fa3d9200640 1 -- 172.21.15.40:0/2463756317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d41573a0 con 0x7fa3d415df30 2024-03-08T06:03:59.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.096+0000 7fa3d9200640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d415df30 0x7fa3d4151c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc036040 tx=0x7fa3cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:59.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.096+0000 7fa3cbe00640 1 -- 172.21.15.40:0/2463756317 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc043020 con 0x7fa3d415df30 2024-03-08T06:03:59.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.096+0000 7fa3cbe00640 1 -- 172.21.15.40:0/2463756317 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc03e450 con 0x7fa3d415df30 2024-03-08T06:03:59.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.096+0000 7fa3db200640 1 -- 172.21.15.40:0/2463756317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d4157680 con 0x7fa3d415df30 2024-03-08T06:03:59.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.096+0000 7fa3cbe00640 1 -- 172.21.15.40:0/2463756317 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc009510 con 0x7fa3d415df30 2024-03-08T06:03:59.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.096+0000 7fa3db200640 1 -- 172.21.15.40:0/2463756317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d4151220 con 0x7fa3d415df30 2024-03-08T06:03:59.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.098+0000 7fa3db200640 1 -- 172.21.15.40:0/2463756317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d4151420 con 0x7fa3d415df30 2024-03-08T06:03:59.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.099+0000 7fa3cbe00640 1 -- 172.21.15.40:0/2463756317 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fa3cc046020 con 0x7fa3d415df30 2024-03-08T06:03:59.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.099+0000 7fa3cbe00640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa3b406b440 0x7fa3b406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.100+0000 7fa3cbe00640 1 -- 172.21.15.40:0/2463756317 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa3cc0ac650 con 0x7fa3d415df30 2024-03-08T06:03:59.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.100+0000 7fa3d3e00640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa3b406b440 0x7fa3b406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.100+0000 7fa3d3e00640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa3b406b440 0x7fa3b406d900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4006fd0 tx=0x7fa3c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:59.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.104+0000 7fa3cbe00640 1 -- 172.21.15.40:0/2463756317 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3cc04d030 con 0x7fa3d415df30 2024-03-08T06:03:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.305+0000 7fa3db200640 1 -- 172.21.15.40:0/2463756317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa3d40c6c30 con 0x7fa3d415df30 2024-03-08T06:03:59.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.306+0000 7fa3cbe00640 1 -- 172.21.15.40:0/2463756317 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa3cc07b870 con 0x7fa3d415df30 2024-03-08T06:03:59.306 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:03:59.306 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-03-08T06:03:59.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:03:59.306 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-03-08T06:03:59.306 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-03-08T06:03:59.306 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:03:59.306 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-03-08T06:03:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.308+0000 7fa3db200640 1 -- 172.21.15.40:0/2463756317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa3b406b440 msgr2=0x7fa3b406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.308+0000 7fa3db200640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa3b406b440 0x7fa3b406d900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4006fd0 tx=0x7fa3c4008040 comp rx=0 tx=0).stop 2024-03-08T06:03:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.308+0000 7fa3db200640 1 -- 172.21.15.40:0/2463756317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d415df30 msgr2=0x7fa3d4151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.308+0000 7fa3db200640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d415df30 0x7fa3d4151c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc036040 tx=0x7fa3cc004a40 comp rx=0 tx=0).stop 2024-03-08T06:03:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.309+0000 7fa3db200640 1 -- 172.21.15.40:0/2463756317 shutdown_connections 2024-03-08T06:03:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.309+0000 7fa3db200640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa3b406b440 0x7fa3b406d900 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.309+0000 7fa3db200640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d415df30 0x7fa3d4151c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.309+0000 7fa3db200640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d40cefc0 0x7fa3d4151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.309+0000 7fa3db200640 1 --2- 172.21.15.40:0/2463756317 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa3d40b9790 0x7fa3d40c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.309+0000 7fa3db200640 1 -- 172.21.15.40:0/2463756317 >> 172.21.15.40:0/2463756317 conn(0x7fa3d406c600 msgr2=0x7fa3d4153ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:59.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.309+0000 7fa3db200640 1 -- 172.21.15.40:0/2463756317 shutdown_connections 2024-03-08T06:03:59.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.309+0000 7fa3db200640 1 -- 172.21.15.40:0/2463756317 wait complete. 2024-03-08T06:03:59.329 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-08T06:03:59.329 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-08T06:03:59.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.556+0000 7fe14f800640 1 Processor -- start 2024-03-08T06:03:59.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.556+0000 7fe14f800640 1 -- start start 2024-03-08T06:03:59.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.558+0000 7fe14f800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe1480b3810 0x7fe1480cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.558+0000 7fe14f800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14806d820 con 0x7fe148071950 2024-03-08T06:03:59.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.558+0000 7fe14f800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14806db00 con 0x7fe148071d30 2024-03-08T06:03:59.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.558+0000 7fe14f800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14806dde0 con 0x7fe1480b3810 2024-03-08T06:03:59.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.558+0000 7fe14ce00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe148071950 0x7fe14806e8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41984/0 (socket says 172.21.15.40:41984) 2024-03-08T06:03:59.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.558+0000 7fe14ce00640 1 -- 172.21.15.40:0/3504573812 learned_addr learned my addr 172.21.15.40:0/3504573812 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:03:59.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.558+0000 7fe14d800640 1 --2- 172.21.15.40:0/3504573812 >> v2:172.21.15.53:3300/0 conn(0x7fe1480b3810 0x7fe1480cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.558+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340480857 0 0) 0x7fe14806d820 con 0x7fe148071950 2024-03-08T06:03:59.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe130003520 con 0x7fe148071950 2024-03-08T06:03:59.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2468358170 0 0) 0x7fe130003520 con 0x7fe148071950 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 >> v2:172.21.15.53:3300/0 conn(0x7fe1480b3810 msgr2=0x7fe1480cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 --2- 172.21.15.40:0/3504573812 >> v2:172.21.15.53:3300/0 conn(0x7fe1480b3810 0x7fe1480cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 >> v1:172.21.15.53:6789/0 conn(0x7fe148071d30 legacy=0x7fe1480b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14806e0c0 con 0x7fe148071950 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fe13c001670 con 0x7fe148071950 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 >> v1:172.21.15.40:6789/0 conn(0x7fe148071950 legacy=0x7fe14806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 --2- 172.21.15.40:0/3504573812 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1300039b0 0x7fe130003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 --2- 172.21.15.40:0/3504573812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe130004800 0x7fe130004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 --2- 172.21.15.40:0/3504573812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1300051c0 0x7fe13000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14806dde0 con 0x7fe130004800 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13000e270 con 0x7fe1300051c0 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13000e470 con 0x7fe1300039b0 2024-03-08T06:03:59.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.559+0000 7fe147e00640 1 --2- 172.21.15.40:0/3504573812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe130004800 0x7fe130004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.560+0000 7fe14d800640 1 --2- 172.21.15.40:0/3504573812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1300051c0 0x7fe13000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.560+0000 7fe147e00640 1 -- 172.21.15.40:0/3504573812 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1300039b0 msgr2=0x7fe130003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:59.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.560+0000 7fe147e00640 1 --2- 172.21.15.40:0/3504573812 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1300039b0 0x7fe130003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.560+0000 7fe147e00640 1 -- 172.21.15.40:0/3504573812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1300051c0 msgr2=0x7fe13000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:03:59.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.560+0000 7fe147e00640 1 --2- 172.21.15.40:0/3504573812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1300051c0 0x7fe13000dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.560+0000 7fe147e00640 1 -- 172.21.15.40:0/3504573812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe130003520 con 0x7fe130004800 2024-03-08T06:03:59.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.560+0000 7fe14d800640 1 -- 172.21.15.40:0/3504573812 reap_dead start 2024-03-08T06:03:59.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.561+0000 7fe147e00640 1 --2- 172.21.15.40:0/3504573812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe130004800 0x7fe130004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe134001750 tx=0x7fe130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e94d48c3fe1f2f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:59.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.561+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134001370 con 0x7fe130004800 2024-03-08T06:03:59.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.561+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1340052d0 con 0x7fe130004800 2024-03-08T06:03:59.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.561+0000 7fe147400640 1 -- 172.21.15.40:0/3504573812 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134003830 con 0x7fe130004800 2024-03-08T06:03:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.562+0000 7fe14f800640 1 -- 172.21.15.40:0/3504573812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe130004800 msgr2=0x7fe130004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.562+0000 7fe14f800640 1 --2- 172.21.15.40:0/3504573812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe130004800 0x7fe130004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe134001750 tx=0x7fe130005830 comp rx=0 tx=0).stop 2024-03-08T06:03:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.562+0000 7fe14f800640 1 -- 172.21.15.40:0/3504573812 shutdown_connections 2024-03-08T06:03:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.562+0000 7fe14f800640 1 --2- 172.21.15.40:0/3504573812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe130004800 0x7fe130004c80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.562+0000 7fe14f800640 1 -- 172.21.15.40:0/3504573812 >> 172.21.15.40:0/3504573812 conn(0x7fe14806ae30 msgr2=0x7fe14806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:03:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.562+0000 7fe14f800640 1 -- 172.21.15.40:0/3504573812 shutdown_connections 2024-03-08T06:03:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.562+0000 7fe14f800640 1 -- 172.21.15.40:0/3504573812 wait complete. 2024-03-08T06:03:59.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.564+0000 7fe14f800640 1 Processor -- start 2024-03-08T06:03:59.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.564+0000 7fe14f800640 1 -- start start 2024-03-08T06:03:59.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.565+0000 7fe14f800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe130004800 0x7fe1481f0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.565+0000 7fe14f800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14813efc0 0x7fe14813c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.565+0000 7fe14f800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1481581b0 0x7fe148158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.565+0000 7fe14f800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe148140a90 con 0x7fe14813efc0 2024-03-08T06:03:59.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.565+0000 7fe14f800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe148140910 con 0x7fe130004800 2024-03-08T06:03:59.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.565+0000 7fe14f800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1480b3100 con 0x7fe1481581b0 2024-03-08T06:03:59.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.565+0000 7fe147e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14813efc0 0x7fe14813c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.565+0000 7fe147e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14813efc0 0x7fe14813c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59772/0 (socket says 172.21.15.40:59772) 2024-03-08T06:03:59.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.565+0000 7fe147e00640 1 -- 172.21.15.40:0/2754900797 learned_addr learned my addr 172.21.15.40:0/2754900797 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:03:59.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.566+0000 7fe14ce00640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe130004800 0x7fe1481f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.566+0000 7fe14d800640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1481581b0 0x7fe148158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.566+0000 7fe147e00640 1 -- 172.21.15.40:0/2754900797 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1481581b0 msgr2=0x7fe148158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:59.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.566+0000 7fe147e00640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1481581b0 0x7fe148158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.566+0000 7fe147e00640 1 -- 172.21.15.40:0/2754900797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe130004800 msgr2=0x7fe1481f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:03:59.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.566+0000 7fe147e00640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe130004800 0x7fe1481f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:03:59.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.566+0000 7fe147e00640 1 -- 172.21.15.40:0/2754900797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480c7700 con 0x7fe14813efc0 2024-03-08T06:03:59.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.566+0000 7fe147e00640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14813efc0 0x7fe14813c420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe134000c00 tx=0x7fe134001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:59.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.566+0000 7fe144c00640 1 -- 172.21.15.40:0/2754900797 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134041020 con 0x7fe14813efc0 2024-03-08T06:03:59.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.566+0000 7fe144c00640 1 -- 172.21.15.40:0/2754900797 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe134005480 con 0x7fe14813efc0 2024-03-08T06:03:59.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.566+0000 7fe144c00640 1 -- 172.21.15.40:0/2754900797 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134007a30 con 0x7fe14813efc0 2024-03-08T06:03:59.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.567+0000 7fe14f800640 1 -- 172.21.15.40:0/2754900797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1480c79e0 con 0x7fe14813efc0 2024-03-08T06:03:59.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.567+0000 7fe14f800640 1 -- 172.21.15.40:0/2754900797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1480c7fa0 con 0x7fe14813efc0 2024-03-08T06:03:59.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.568+0000 7fe14f800640 1 -- 172.21.15.40:0/2754900797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe148067cb0 con 0x7fe14813efc0 2024-03-08T06:03:59.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.569+0000 7fe144c00640 1 -- 172.21.15.40:0/2754900797 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fe134044020 con 0x7fe14813efc0 2024-03-08T06:03:59.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.570+0000 7fe144c00640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12806b440 0x7fe12806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:03:59.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.570+0000 7fe144c00640 1 -- 172.21.15.40:0/2754900797 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe1340ab720 con 0x7fe14813efc0 2024-03-08T06:03:59.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.570+0000 7fe14ce00640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12806b440 0x7fe12806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:03:59.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.571+0000 7fe14ce00640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12806b440 0x7fe12806d900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe13c001860 tx=0x7fe13c0015f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:03:59.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.574+0000 7fe144c00640 1 -- 172.21.15.40:0/2754900797 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe13404b030 con 0x7fe14813efc0 2024-03-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:03:59.772+0000 7fe14f800640 1 -- 172.21.15.40:0/2754900797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe148143720 con 0x7fe14813efc0 2024-03-08T06:04:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.496+0000 7fe144c00640 1 -- 172.21.15.40:0/2754900797 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe134006e20 con 0x7fe14813efc0 2024-03-08T06:04:00.513 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-03-08T06:04:00.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.501+0000 7fe14f800640 1 -- 172.21.15.40:0/2754900797 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12806b440 msgr2=0x7fe12806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:00.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.501+0000 7fe14f800640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12806b440 0x7fe12806d900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe13c001860 tx=0x7fe13c0015f0 comp rx=0 tx=0).stop 2024-03-08T06:04:00.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.501+0000 7fe14f800640 1 -- 172.21.15.40:0/2754900797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14813efc0 msgr2=0x7fe14813c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:00.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.501+0000 7fe14f800640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14813efc0 0x7fe14813c420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe134000c00 tx=0x7fe134001120 comp rx=0 tx=0).stop 2024-03-08T06:04:00.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.501+0000 7fe14f800640 1 -- 172.21.15.40:0/2754900797 shutdown_connections 2024-03-08T06:04:00.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.501+0000 7fe14f800640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12806b440 0x7fe12806d900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:00.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.501+0000 7fe14f800640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1481581b0 0x7fe148158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:00.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.501+0000 7fe14f800640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14813efc0 0x7fe14813c420 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe134000c00 tx=0x7fe134001120 comp rx=0 tx=0).stop 2024-03-08T06:04:00.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.501+0000 7fe14f800640 1 --2- 172.21.15.40:0/2754900797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe130004800 0x7fe1481f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:00.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.501+0000 7fe14f800640 1 -- 172.21.15.40:0/2754900797 >> 172.21.15.40:0/2754900797 conn(0x7fe14806ae30 msgr2=0x7fe148159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:00.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.501+0000 7fe14f800640 1 -- 172.21.15.40:0/2754900797 shutdown_connections 2024-03-08T06:04:00.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.502+0000 7fe14f800640 1 -- 172.21.15.40:0/2754900797 wait complete. 2024-03-08T06:04:00.533 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-08T06:04:00.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.735+0000 7fcad3e00640 1 Processor -- start 2024-03-08T06:04:00.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.736+0000 7fcad3e00640 1 -- start start 2024-03-08T06:04:00.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.737+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc15de50 0x7fcacc161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:00.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.737+0000 7fcad3e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcacc0cf240 0x7fcacc0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:00.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.737+0000 7fcad3e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc162520 con 0x7fcacc0b97d0 2024-03-08T06:04:00.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.737+0000 7fcad3e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcacc162ae0 con 0x7fcacc0cf240 2024-03-08T06:04:00.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.737+0000 7fcad3e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0d2360 con 0x7fcacc15de50 2024-03-08T06:04:00.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.738+0000 7fcad0a00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcacc0b97d0 0x7fcacc0ceb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41992/0 (socket says 172.21.15.40:41992) 2024-03-08T06:04:00.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.738+0000 7fcad0a00640 1 -- 172.21.15.40:0/2919712076 learned_addr learned my addr 172.21.15.40:0/2919712076 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:00.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.738+0000 7fcad1e00640 1 --2- 172.21.15.40:0/2919712076 >> v2:172.21.15.53:3300/0 conn(0x7fcacc0cf240 0x7fcacc0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:00.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.738+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334629208 0 0) 0x7fcacc162520 con 0x7fcacc0b97d0 2024-03-08T06:04:00.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.738+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcab4003520 con 0x7fcacc0b97d0 2024-03-08T06:04:00.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1601903617 0 0) 0x7fcab4003520 con 0x7fcacc0b97d0 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc15de50 msgr2=0x7fcacc161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcad1400640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc15de50 0x7fcacc161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc15de50 0x7fcacc161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 >> v2:172.21.15.53:3300/0 conn(0x7fcacc0cf240 msgr2=0x7fcacc0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 --2- 172.21.15.40:0/2919712076 >> v2:172.21.15.53:3300/0 conn(0x7fcacc0cf240 0x7fcacc0d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc162800 con 0x7fcacc0b97d0 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fcab8001690 con 0x7fcacc0b97d0 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 >> v1:172.21.15.40:6789/0 conn(0x7fcacc0b97d0 legacy=0x7fcacc0ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc15de50 0x7fcab40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 0x7fcab4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcab4004d40 0x7fcab40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc162ae0 con 0x7fcab4003f30 2024-03-08T06:04:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0d2360 con 0x7fcab4004d40 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab400baf0 con 0x7fcacc15de50 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcad1e00640 1 --2- 172.21.15.40:0/2919712076 >> v2:172.21.15.53:3300/0 conn(0x7fcacc0cf240 0x7fcacc0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcad0a00640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 0x7fcab4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcad1400640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc15de50 0x7fcab40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.739+0000 7fcad1e00640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcab4004d40 0x7fcab40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.740+0000 7fcad0a00640 1 -- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc15de50 msgr2=0x7fcab40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.740+0000 7fcad0a00640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc15de50 0x7fcab40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.740+0000 7fcad0a00640 1 -- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcab4004d40 msgr2=0x7fcab40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.740+0000 7fcad0a00640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcab4004d40 0x7fcab40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.740+0000 7fcad0a00640 1 -- 172.21.15.40:0/2919712076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4003520 con 0x7fcab4003f30 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.740+0000 7fcad1e00640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcab4004d40 0x7fcab40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.741+0000 7fcad0a00640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 0x7fcab4004800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcab8007fd0 tx=0x7fcab4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d92401fc31c183 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:00.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.741+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8007d50 con 0x7fcab4003f30 2024-03-08T06:04:00.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.741+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab8030070 con 0x7fcab4003f30 2024-03-08T06:04:00.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.741+0000 7fcacbe00640 1 -- 172.21.15.40:0/2919712076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8005d10 con 0x7fcab4003f30 2024-03-08T06:04:00.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.742+0000 7fcad3e00640 1 -- 172.21.15.40:0/2919712076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 msgr2=0x7fcab4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:00.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.742+0000 7fcad3e00640 1 --2- 172.21.15.40:0/2919712076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 0x7fcab4004800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcab8007fd0 tx=0x7fcab4005830 comp rx=0 tx=0).stop 2024-03-08T06:04:00.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.742+0000 7fcad1e00640 1 -- 172.21.15.40:0/2919712076 reap_dead start 2024-03-08T06:04:00.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.742+0000 7fcad3e00640 1 -- 172.21.15.40:0/2919712076 shutdown_connections 2024-03-08T06:04:00.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.742+0000 7fcad3e00640 1 -- 172.21.15.40:0/2919712076 >> 172.21.15.40:0/2919712076 conn(0x7fcacc06c490 msgr2=0x7fcacc06c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:00.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.742+0000 7fcad3e00640 1 -- 172.21.15.40:0/2919712076 shutdown_connections 2024-03-08T06:04:00.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.742+0000 7fcad3e00640 1 -- 172.21.15.40:0/2919712076 wait complete. 2024-03-08T06:04:00.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.743+0000 7fcad3e00640 1 Processor -- start 2024-03-08T06:04:00.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.744+0000 7fcad3e00640 1 -- start start 2024-03-08T06:04:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 0x7fcacc0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc1f6fb0 0x7fcacc1f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc1f3210 0x7fcacc1f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad3e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc15e3b0 con 0x7fcab4003f30 2024-03-08T06:04:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc15e230 con 0x7fcacc1f6fb0 2024-03-08T06:04:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad3e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc15e530 con 0x7fcacc1f3210 2024-03-08T06:04:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad1400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 0x7fcacc0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad1400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 0x7fcacc0ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59794/0 (socket says 172.21.15.40:59794) 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad1400640 1 -- 172.21.15.40:0/792587700 learned_addr learned my addr 172.21.15.40:0/792587700 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad0a00640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc1f6fb0 0x7fcacc1f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad1e00640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc1f3210 0x7fcacc1f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad1400640 1 -- 172.21.15.40:0/792587700 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc1f3210 msgr2=0x7fcacc1f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad1400640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc1f3210 0x7fcacc1f3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad1400640 1 -- 172.21.15.40:0/792587700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc1f6fb0 msgr2=0x7fcacc1f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad1400640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc1f6fb0 0x7fcacc1f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad1400640 1 -- 172.21.15.40:0/792587700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc1f3bd0 con 0x7fcab4003f30 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.745+0000 7fcad1e00640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc1f3210 0x7fcacc1f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.746+0000 7fcad1400640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 0x7fcacc0ccc10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcac000eef0 tx=0x7fcac000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.746+0000 7fcac9600640 1 -- 172.21.15.40:0/792587700 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac000f810 con 0x7fcab4003f30 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.746+0000 7fcac9600640 1 -- 172.21.15.40:0/792587700 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac000f9b0 con 0x7fcab4003f30 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.746+0000 7fcac9600640 1 -- 172.21.15.40:0/792587700 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac000fda0 con 0x7fcab4003f30 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.746+0000 7fcad3e00640 1 -- 172.21.15.40:0/792587700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc0aa9b0 con 0x7fcab4003f30 2024-03-08T06:04:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.746+0000 7fcad3e00640 1 -- 172.21.15.40:0/792587700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc0aaf70 con 0x7fcab4003f30 2024-03-08T06:04:00.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.747+0000 7fcad3e00640 1 -- 172.21.15.40:0/792587700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc0687d0 con 0x7fcab4003f30 2024-03-08T06:04:00.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.748+0000 7fcac9600640 1 -- 172.21.15.40:0/792587700 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcac001a480 con 0x7fcab4003f30 2024-03-08T06:04:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.749+0000 7fcac9600640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcaac06b4d0 0x7fcaac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.749+0000 7fcac9600640 1 -- 172.21.15.40:0/792587700 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fcac008b230 con 0x7fcab4003f30 2024-03-08T06:04:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.749+0000 7fcad0a00640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcaac06b4d0 0x7fcaac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:00.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.750+0000 7fcad0a00640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcaac06b4d0 0x7fcaac06d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcab8034000 tx=0x7fcab8039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:00.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.754+0000 7fcac9600640 1 -- 172.21.15.40:0/792587700 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcacc0687d0 con 0x7fcab4003f30 2024-03-08T06:04:00.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:00.960+0000 7fcad3e00640 1 -- 172.21.15.40:0/792587700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcacc067cb0 con 0x7fcab4003f30 2024-03-08T06:04:01.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.513+0000 7fcac9600640 1 -- 172.21.15.40:0/792587700 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcac005a1f0 con 0x7fcab4003f30 2024-03-08T06:04:01.514 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-03-08T06:04:01.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.517+0000 7fcad3e00640 1 -- 172.21.15.40:0/792587700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcaac06b4d0 msgr2=0x7fcaac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.517+0000 7fcad3e00640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcaac06b4d0 0x7fcaac06d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcab8034000 tx=0x7fcab8039040 comp rx=0 tx=0).stop 2024-03-08T06:04:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.517+0000 7fcad3e00640 1 -- 172.21.15.40:0/792587700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 msgr2=0x7fcacc0ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.517+0000 7fcad3e00640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 0x7fcacc0ccc10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcac000eef0 tx=0x7fcac000c6e0 comp rx=0 tx=0).stop 2024-03-08T06:04:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.517+0000 7fcad3e00640 1 -- 172.21.15.40:0/792587700 shutdown_connections 2024-03-08T06:04:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.517+0000 7fcad3e00640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcaac06b4d0 0x7fcaac06d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.517+0000 7fcad3e00640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcacc1f3210 0x7fcacc1f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.517+0000 7fcad3e00640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc1f6fb0 0x7fcacc1f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.517+0000 7fcad3e00640 1 --2- 172.21.15.40:0/792587700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab4003f30 0x7fcacc0ccc10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.517+0000 7fcad3e00640 1 -- 172.21.15.40:0/792587700 >> 172.21.15.40:0/792587700 conn(0x7fcacc06c490 msgr2=0x7fcacc074510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:01.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.518+0000 7fcad3e00640 1 -- 172.21.15.40:0/792587700 shutdown_connections 2024-03-08T06:04:01.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.518+0000 7fcad3e00640 1 -- 172.21.15.40:0/792587700 wait complete. 2024-03-08T06:04:01.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-08T06:04:01.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.742+0000 7f5e1ea00640 1 Processor -- start 2024-03-08T06:04:01.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.742+0000 7f5e1ea00640 1 -- start start 2024-03-08T06:04:01.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.743+0000 7f5e1ea00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e18145590 0x7f5e180cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:01.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.744+0000 7f5e1ea00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e18160610 con 0x7f5e180b3bd0 2024-03-08T06:04:01.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.744+0000 7f5e1ea00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e18071990 con 0x7f5e180ccc70 2024-03-08T06:04:01.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.744+0000 7f5e1ea00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18071c70 con 0x7f5e18145590 2024-03-08T06:04:01.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.744+0000 7f5e1ca00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5e180b3bd0 0x7f5e1815ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41998/0 (socket says 172.21.15.40:41998) 2024-03-08T06:04:01.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.744+0000 7f5e1ca00640 1 -- 172.21.15.40:0/1969900671 learned_addr learned my addr 172.21.15.40:0/1969900671 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.744+0000 7f5e17e00640 1 --2- 172.21.15.40:0/1969900671 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e18145590 0x7f5e180cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.744+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653713714 0 0) 0x7f5e18160610 con 0x7f5e180b3bd0 2024-03-08T06:04:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.744+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dfc003520 con 0x7f5e180b3bd0 2024-03-08T06:04:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3490136416 0 0) 0x7f5dfc003520 con 0x7f5e180b3bd0 2024-03-08T06:04:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e18145590 msgr2=0x7f5e180cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 --2- 172.21.15.40:0/1969900671 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e18145590 0x7f5e180cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 >> v1:172.21.15.53:6789/0 conn(0x7f5e180ccc70 legacy=0x7f5e1815f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:01.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e18071f50 con 0x7f5e180b3bd0 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f5e100019f0 con 0x7f5e180b3bd0 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 >> v1:172.21.15.40:6789/0 conn(0x7f5e180b3bd0 legacy=0x7f5e1815ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 --2- 172.21.15.40:0/1969900671 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e18145590 0x7f5dfc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 --2- 172.21.15.40:0/1969900671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dfc003f60 0x7f5dfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 --2- 172.21.15.40:0/1969900671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dfc004d40 0x7f5dfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18071c70 con 0x7f5dfc004d40 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc00baa0 con 0x7f5dfc003f60 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc00bca0 con 0x7f5e18145590 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.745+0000 7f5e1ca00640 1 --2- 172.21.15.40:0/1969900671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dfc004d40 0x7f5dfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.746+0000 7f5e17400640 1 --2- 172.21.15.40:0/1969900671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dfc003f60 0x7f5dfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.746+0000 7f5e1ca00640 1 -- 172.21.15.40:0/1969900671 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e18145590 msgr2=0x7f5dfc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.746+0000 7f5e1ca00640 1 --2- 172.21.15.40:0/1969900671 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e18145590 0x7f5dfc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.746+0000 7f5e1ca00640 1 -- 172.21.15.40:0/1969900671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dfc003f60 msgr2=0x7f5dfc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.746+0000 7f5e1ca00640 1 --2- 172.21.15.40:0/1969900671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dfc003f60 0x7f5dfc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:01.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.746+0000 7f5e1ca00640 1 -- 172.21.15.40:0/1969900671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc003520 con 0x7f5dfc004d40 2024-03-08T06:04:01.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.747+0000 7f5e1ca00640 1 --2- 172.21.15.40:0/1969900671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dfc004d40 0x7f5dfc0051c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000a850 tx=0x7f5dfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e97d350a9070074 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.747+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000a4a0 con 0x7f5dfc004d40 2024-03-08T06:04:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.747+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10034650 con 0x7f5dfc004d40 2024-03-08T06:04:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.747+0000 7f5e16a00640 1 -- 172.21.15.40:0/1969900671 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10034a80 con 0x7f5dfc004d40 2024-03-08T06:04:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.748+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1969900671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dfc004d40 msgr2=0x7f5dfc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.748+0000 7f5e1ea00640 1 --2- 172.21.15.40:0/1969900671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dfc004d40 0x7f5dfc0051c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000a850 tx=0x7f5dfc005830 comp rx=0 tx=0).stop 2024-03-08T06:04:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.748+0000 7f5e1ca00640 1 -- 172.21.15.40:0/1969900671 reap_dead start 2024-03-08T06:04:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.748+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1969900671 shutdown_connections 2024-03-08T06:04:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.748+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1969900671 >> 172.21.15.40:0/1969900671 conn(0x7f5e180cfaa0 msgr2=0x7f5e180cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.748+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1969900671 shutdown_connections 2024-03-08T06:04:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.748+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1969900671 wait complete. 2024-03-08T06:04:01.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.750+0000 7f5e1ea00640 1 Processor -- start 2024-03-08T06:04:01.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.750+0000 7f5e1ea00640 1 -- start start 2024-03-08T06:04:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dfc004d40 0x7f5e181e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ea00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e1815e250 0x7f5e1815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e181440d0 0x7f5e18144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ea00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18145970 con 0x7f5e181440d0 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18072510 con 0x7f5dfc004d40 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ea00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180cd960 con 0x7f5e1815e250 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ca00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e181440d0 0x7f5e18144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ca00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e181440d0 0x7f5e18144550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59806/0 (socket says 172.21.15.40:59806) 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ca00640 1 -- 172.21.15.40:0/1579856025 learned_addr learned my addr 172.21.15.40:0/1579856025 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e17e00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dfc004d40 0x7f5e181e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e17400640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e1815e250 0x7f5e1815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ca00640 1 -- 172.21.15.40:0/1579856025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e1815e250 msgr2=0x7f5e1815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ca00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e1815e250 0x7f5e1815e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ca00640 1 -- 172.21.15.40:0/1579856025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dfc004d40 msgr2=0x7f5e181e6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ca00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dfc004d40 0x7f5e181e6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e1ca00640 1 -- 172.21.15.40:0/1579856025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e18144a90 con 0x7f5e181440d0 2024-03-08T06:04:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.751+0000 7f5e17400640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e1815e250 0x7f5e1815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.752+0000 7f5e1ca00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e181440d0 0x7f5e18144550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5e10001cf0 tx=0x7f5e10001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.752+0000 7f5e0fe00640 1 -- 172.21.15.40:0/1579856025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000a480 con 0x7f5e181440d0 2024-03-08T06:04:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.752+0000 7f5e0fe00640 1 -- 172.21.15.40:0/1579856025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10044070 con 0x7f5e181440d0 2024-03-08T06:04:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.752+0000 7f5e0fe00640 1 -- 172.21.15.40:0/1579856025 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10034a00 con 0x7f5e181440d0 2024-03-08T06:04:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.752+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1579856025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1815ed10 con 0x7f5e181440d0 2024-03-08T06:04:01.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.752+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1579856025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1815f2d0 con 0x7f5e181440d0 2024-03-08T06:04:01.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.754+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1579856025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e18067cb0 con 0x7f5e181440d0 2024-03-08T06:04:01.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.755+0000 7f5e0fe00640 1 -- 172.21.15.40:0/1579856025 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e10034ba0 con 0x7f5e181440d0 2024-03-08T06:04:01.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.755+0000 7f5e0fe00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5df806b500 0x7f5df806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:01.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.756+0000 7f5e0fe00640 1 -- 172.21.15.40:0/1579856025 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5e100ac390 con 0x7f5e181440d0 2024-03-08T06:04:01.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.756+0000 7f5e17e00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5df806b500 0x7f5df806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:01.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.756+0000 7f5e17e00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5df806b500 0x7f5df806d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e08004770 tx=0x7f5e08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:01.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:01.761+0000 7f5e0fe00640 1 -- 172.21.15.40:0/1579856025 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e1007b300 con 0x7f5e181440d0 2024-03-08T06:04:02.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.001+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1579856025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5e180aec30 con 0x7f5e181440d0 2024-03-08T06:04:02.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.529+0000 7f5e0fe00640 1 -- 172.21.15.40:0/1579856025 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5e10032e30 con 0x7f5e181440d0 2024-03-08T06:04:02.532 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 2 and data pool 3 2024-03-08T06:04:02.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.539+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1579856025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5df806b500 msgr2=0x7f5df806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:02.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.539+0000 7f5e1ea00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5df806b500 0x7f5df806d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e08004770 tx=0x7f5e08008040 comp rx=0 tx=0).stop 2024-03-08T06:04:02.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.539+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1579856025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e181440d0 msgr2=0x7f5e18144550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:02.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.539+0000 7f5e1ea00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e181440d0 0x7f5e18144550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5e10001cf0 tx=0x7f5e10001dc0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.540+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1579856025 shutdown_connections 2024-03-08T06:04:02.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.540+0000 7f5e1ea00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5df806b500 0x7f5df806d9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.540+0000 7f5e1ea00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e181440d0 0x7f5e18144550 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.540+0000 7f5e1ea00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e1815e250 0x7f5e1815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.540+0000 7f5e1ea00640 1 --2- 172.21.15.40:0/1579856025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dfc004d40 0x7f5e181e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.540+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1579856025 >> 172.21.15.40:0/1579856025 conn(0x7f5e180cfaa0 msgr2=0x7f5e1806ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:02.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.540+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1579856025 shutdown_connections 2024-03-08T06:04:02.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.540+0000 7f5e1ea00640 1 -- 172.21.15.40:0/1579856025 wait complete. 2024-03-08T06:04:02.579 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.759+0000 7fe75aa00640 1 Processor -- start 2024-03-08T06:04:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.760+0000 7fe75aa00640 1 -- start start 2024-03-08T06:04:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.761+0000 7fe75aa00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe75415bea0 0x7fe75415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.761+0000 7fe75aa00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe754071df0 0x7fe75406d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.761+0000 7fe75aa00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75415ceb0 con 0x7fe754145520 2024-03-08T06:04:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.761+0000 7fe75aa00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75415c330 con 0x7fe75415bea0 2024-03-08T06:04:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.761+0000 7fe75aa00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75415c610 con 0x7fe754071df0 2024-03-08T06:04:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.761+0000 7fe758c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe75415bea0 0x7fe75415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.762+0000 7fe758c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe75415bea0 0x7fe75415d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59808/0 (socket says 172.21.15.40:59808) 2024-03-08T06:04:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.762+0000 7fe758c00640 1 -- 172.21.15.40:0/2829008009 learned_addr learned my addr 172.21.15.40:0/2829008009 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.762+0000 7fe75a000640 1 --2- 172.21.15.40:0/2829008009 >> v2:172.21.15.53:3300/0 conn(0x7fe754071df0 0x7fe75406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.762+0000 7fe758c00640 1 -- 172.21.15.40:0/2829008009 >> v2:172.21.15.53:3300/0 conn(0x7fe754071df0 msgr2=0x7fe75406d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.762+0000 7fe758c00640 1 --2- 172.21.15.40:0/2829008009 >> v2:172.21.15.53:3300/0 conn(0x7fe754071df0 0x7fe75406d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.762+0000 7fe758c00640 1 -- 172.21.15.40:0/2829008009 >> v1:172.21.15.53:6789/0 conn(0x7fe754145520 legacy=0x7fe7540716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.762+0000 7fe758c00640 1 -- 172.21.15.40:0/2829008009 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75415c8f0 con 0x7fe75415bea0 2024-03-08T06:04:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.763+0000 7fe758c00640 1 --2- 172.21.15.40:0/2829008009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75415bea0 0x7fe75415d0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe754071260 tx=0x7fe74002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9124767d9d5a717 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.763+0000 7fe753e00640 1 -- 172.21.15.40:0/2829008009 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740034070 con 0x7fe75415bea0 2024-03-08T06:04:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.764+0000 7fe753e00640 1 -- 172.21.15.40:0/2829008009 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe740030860 con 0x7fe75415bea0 2024-03-08T06:04:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.764+0000 7fe753e00640 1 -- 172.21.15.40:0/2829008009 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740030c30 con 0x7fe75415bea0 2024-03-08T06:04:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.764+0000 7fe75aa00640 1 -- 172.21.15.40:0/2829008009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75415bea0 msgr2=0x7fe75415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.764+0000 7fe75aa00640 1 --2- 172.21.15.40:0/2829008009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75415bea0 0x7fe75415d0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe754071260 tx=0x7fe74002b9a0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.764+0000 7fe75aa00640 1 -- 172.21.15.40:0/2829008009 shutdown_connections 2024-03-08T06:04:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.764+0000 7fe75aa00640 1 --2- 172.21.15.40:0/2829008009 >> v2:172.21.15.53:3300/0 conn(0x7fe754071df0 0x7fe75406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.764+0000 7fe75aa00640 1 --2- 172.21.15.40:0/2829008009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75415bea0 0x7fe75415d0a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.764+0000 7fe75aa00640 1 -- 172.21.15.40:0/2829008009 >> 172.21.15.40:0/2829008009 conn(0x7fe7540b1850 msgr2=0x7fe7540b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.764+0000 7fe75aa00640 1 -- 172.21.15.40:0/2829008009 shutdown_connections 2024-03-08T06:04:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.764+0000 7fe75aa00640 1 -- 172.21.15.40:0/2829008009 wait complete. 2024-03-08T06:04:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.766+0000 7fe75aa00640 1 Processor -- start 2024-03-08T06:04:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.766+0000 7fe75aa00640 1 -- start start 2024-03-08T06:04:02.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.767+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe754071df0 0x7fe75414d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:02.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.767+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754145520 0x7fe75413b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:02.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.767+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe754150050 0x7fe75413bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:02.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe75aa00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540af1c0 con 0x7fe754145520 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe75aa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540af040 con 0x7fe754150050 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe75aa00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75406e410 con 0x7fe754071df0 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe758c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754145520 0x7fe75413b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe758c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754145520 0x7fe75413b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59810/0 (socket says 172.21.15.40:59810) 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe758c00640 1 -- 172.21.15.40:0/1859631709 learned_addr learned my addr 172.21.15.40:0/1859631709 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe759600640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe754071df0 0x7fe75414d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe75a000640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe754150050 0x7fe75413bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe758c00640 1 -- 172.21.15.40:0/1859631709 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe754071df0 msgr2=0x7fe75414d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe758c00640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe754071df0 0x7fe75414d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe758c00640 1 -- 172.21.15.40:0/1859631709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe754150050 msgr2=0x7fe75413bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe758c00640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe754150050 0x7fe75413bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe758c00640 1 -- 172.21.15.40:0/1859631709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75413d8c0 con 0x7fe754145520 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.768+0000 7fe75a000640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe754150050 0x7fe75413bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.769+0000 7fe758c00640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754145520 0x7fe75413b7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe740000c00 tx=0x7fe7400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.769+0000 7fe751600640 1 -- 172.21.15.40:0/1859631709 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740034040 con 0x7fe754145520 2024-03-08T06:04:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.769+0000 7fe751600640 1 -- 172.21.15.40:0/1859631709 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe740039d90 con 0x7fe754145520 2024-03-08T06:04:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.769+0000 7fe75aa00640 1 -- 172.21.15.40:0/1859631709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75415d830 con 0x7fe754145520 2024-03-08T06:04:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.769+0000 7fe751600640 1 -- 172.21.15.40:0/1859631709 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740043610 con 0x7fe754145520 2024-03-08T06:04:02.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.769+0000 7fe75aa00640 1 -- 172.21.15.40:0/1859631709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75413db40 con 0x7fe754145520 2024-03-08T06:04:02.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.770+0000 7fe75aa00640 1 -- 172.21.15.40:0/1859631709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe754075100 con 0x7fe754145520 2024-03-08T06:04:02.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.771+0000 7fe751600640 1 -- 172.21.15.40:0/1859631709 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe740037070 con 0x7fe754145520 2024-03-08T06:04:02.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.772+0000 7fe751600640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe73406b4d0 0x7fe73406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:02.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.772+0000 7fe751600640 1 -- 172.21.15.40:0/1859631709 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7400ac360 con 0x7fe754145520 2024-03-08T06:04:02.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.772+0000 7fe759600640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe73406b4d0 0x7fe73406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:02.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.773+0000 7fe759600640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe73406b4d0 0x7fe73406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe7480015a0 tx=0x7fe748009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:02.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.777+0000 7fe751600640 1 -- 172.21.15.40:0/1859631709 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe75413d8c0 con 0x7fe754145520 2024-03-08T06:04:02.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.979+0000 7fe75aa00640 1 -- 172.21.15.40:0/1859631709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe75406c480 con 0x7fe754145520 2024-03-08T06:04:02.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.981+0000 7fe751600640 1 -- 172.21.15.40:0/1859631709 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16534 (secure 0 0 0) 0x7fe74003e070 con 0x7fe754145520 2024-03-08T06:04:02.981 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:02.981 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:02.002825+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:02.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 -- 172.21.15.40:0/1859631709 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe73406b4d0 msgr2=0x7fe73406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:02.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe73406b4d0 0x7fe73406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe7480015a0 tx=0x7fe748009040 comp rx=0 tx=0).stop 2024-03-08T06:04:02.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 -- 172.21.15.40:0/1859631709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754145520 msgr2=0x7fe75413b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:02.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754145520 0x7fe75413b7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe740000c00 tx=0x7fe7400046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 -- 172.21.15.40:0/1859631709 shutdown_connections 2024-03-08T06:04:02.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe73406b4d0 0x7fe73406d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe754150050 0x7fe75413bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754145520 0x7fe75413b7b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 --2- 172.21.15.40:0/1859631709 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe754071df0 0x7fe75414d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 -- 172.21.15.40:0/1859631709 >> 172.21.15.40:0/1859631709 conn(0x7fe7540b1850 msgr2=0x7fe7540b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 -- 172.21.15.40:0/1859631709 shutdown_connections 2024-03-08T06:04:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:02.984+0000 7fe75aa00640 1 -- 172.21.15.40:0/1859631709 wait complete. 2024-03-08T06:04:03.003 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.226+0000 7fdf03200640 1 Processor -- start 2024-03-08T06:04:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.226+0000 7fdf03200640 1 -- start start 2024-03-08T06:04:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.228+0000 7fdf03200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdefc15e580 0x7fdefc06e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.228+0000 7fdf03200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdefc06ec50 0x7fdefc0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.228+0000 7fdf03200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc15eaf0 con 0x7fdefc144030 2024-03-08T06:04:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.228+0000 7fdf03200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc0b3350 con 0x7fdefc06ec50 2024-03-08T06:04:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.228+0000 7fdf03200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc15edd0 con 0x7fdefc15e580 2024-03-08T06:04:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.228+0000 7fdf01200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdefc06ec50 0x7fdefc0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.228+0000 7fdf01200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdefc06ec50 0x7fdefc0b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59814/0 (socket says 172.21.15.40:59814) 2024-03-08T06:04:03.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.228+0000 7fdf01200640 1 -- 172.21.15.40:0/3469859647 learned_addr learned my addr 172.21.15.40:0/3469859647 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:03.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.228+0000 7fdefb400640 1 --2- 172.21.15.40:0/3469859647 >> v2:172.21.15.53:3300/0 conn(0x7fdefc15e580 0x7fdefc06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.229+0000 7fdf01200640 1 -- 172.21.15.40:0/3469859647 >> v2:172.21.15.53:3300/0 conn(0x7fdefc15e580 msgr2=0x7fdefc06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.229+0000 7fdf01200640 1 --2- 172.21.15.40:0/3469859647 >> v2:172.21.15.53:3300/0 conn(0x7fdefc15e580 0x7fdefc06e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.229+0000 7fdf01200640 1 -- 172.21.15.40:0/3469859647 >> v1:172.21.15.40:6789/0 conn(0x7fdefc144030 legacy=0x7fdefc0cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.229+0000 7fdf01200640 1 -- 172.21.15.40:0/3469859647 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc15f0b0 con 0x7fdefc06ec50 2024-03-08T06:04:03.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.230+0000 7fdf01200640 1 --2- 172.21.15.40:0/3469859647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdefc06ec50 0x7fdefc0b37e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdefc15e4e0 tx=0x7fdef402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b5d0e038761b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:03.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.230+0000 7fdefaa00640 1 -- 172.21.15.40:0/3469859647 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef402cb20 con 0x7fdefc06ec50 2024-03-08T06:04:03.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.230+0000 7fdefaa00640 1 -- 172.21.15.40:0/3469859647 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef402ccc0 con 0x7fdefc06ec50 2024-03-08T06:04:03.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.230+0000 7fdefaa00640 1 -- 172.21.15.40:0/3469859647 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4034610 con 0x7fdefc06ec50 2024-03-08T06:04:03.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.231+0000 7fdf03200640 1 -- 172.21.15.40:0/3469859647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdefc06ec50 msgr2=0x7fdefc0b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:03.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.231+0000 7fdf03200640 1 --2- 172.21.15.40:0/3469859647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdefc06ec50 0x7fdefc0b37e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdefc15e4e0 tx=0x7fdef402bf80 comp rx=0 tx=0).stop 2024-03-08T06:04:03.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.231+0000 7fdf03200640 1 -- 172.21.15.40:0/3469859647 shutdown_connections 2024-03-08T06:04:03.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.231+0000 7fdf03200640 1 --2- 172.21.15.40:0/3469859647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdefc06ec50 0x7fdefc0b37e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.231+0000 7fdf03200640 1 --2- 172.21.15.40:0/3469859647 >> v2:172.21.15.53:3300/0 conn(0x7fdefc15e580 0x7fdefc06e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.231+0000 7fdf03200640 1 -- 172.21.15.40:0/3469859647 >> 172.21.15.40:0/3469859647 conn(0x7fdefc06ae30 msgr2=0x7fdefc06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:03.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.231+0000 7fdf03200640 1 -- 172.21.15.40:0/3469859647 shutdown_connections 2024-03-08T06:04:03.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.231+0000 7fdf03200640 1 -- 172.21.15.40:0/3469859647 wait complete. 2024-03-08T06:04:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.233+0000 7fdf03200640 1 Processor -- start 2024-03-08T06:04:03.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.233+0000 7fdf03200640 1 -- start start 2024-03-08T06:04:03.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.234+0000 7fdf03200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdefc06ec50 0x7fdefc15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.234+0000 7fdf03200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdefc144030 0x7fdefc142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.234+0000 7fdf03200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdefc15e580 0x7fdefc142640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.234+0000 7fdf03200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc144590 con 0x7fdefc15e580 2024-03-08T06:04:03.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.234+0000 7fdf03200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc144410 con 0x7fdefc144030 2024-03-08T06:04:03.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.234+0000 7fdf03200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0b3d20 con 0x7fdefc06ec50 2024-03-08T06:04:03.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.234+0000 7fdf01200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdefc15e580 0x7fdefc142640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.234+0000 7fdf01200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdefc15e580 0x7fdefc142640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59818/0 (socket says 172.21.15.40:59818) 2024-03-08T06:04:03.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.234+0000 7fdf01200640 1 -- 172.21.15.40:0/2740539812 learned_addr learned my addr 172.21.15.40:0/2740539812 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:03.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.235+0000 7fdefbe00640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdefc06ec50 0x7fdefc15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.235+0000 7fdefb400640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdefc144030 0x7fdefc142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.235+0000 7fdf01200640 1 -- 172.21.15.40:0/2740539812 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdefc06ec50 msgr2=0x7fdefc15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:03.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.235+0000 7fdf01200640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdefc06ec50 0x7fdefc15a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.235+0000 7fdf01200640 1 -- 172.21.15.40:0/2740539812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdefc144030 msgr2=0x7fdefc142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:03.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.235+0000 7fdf01200640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdefc144030 0x7fdefc142100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.235+0000 7fdf01200640 1 -- 172.21.15.40:0/2740539812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc14a090 con 0x7fdefc15e580 2024-03-08T06:04:03.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.235+0000 7fdefb400640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdefc144030 0x7fdefc142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:03.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.235+0000 7fdf01200640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdefc15e580 0x7fdefc142640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009f40 tx=0x7fdef4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:03.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.236+0000 7fdef3e00640 1 -- 172.21.15.40:0/2740539812 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4043020 con 0x7fdefc15e580 2024-03-08T06:04:03.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.236+0000 7fdef3e00640 1 -- 172.21.15.40:0/2740539812 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef403e450 con 0x7fdefc15e580 2024-03-08T06:04:03.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.236+0000 7fdf03200640 1 -- 172.21.15.40:0/2740539812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc14a370 con 0x7fdefc15e580 2024-03-08T06:04:03.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.236+0000 7fdef3e00640 1 -- 172.21.15.40:0/2740539812 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4009510 con 0x7fdefc15e580 2024-03-08T06:04:03.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.236+0000 7fdf03200640 1 -- 172.21.15.40:0/2740539812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc14a930 con 0x7fdefc15e580 2024-03-08T06:04:03.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.237+0000 7fdf03200640 1 -- 172.21.15.40:0/2740539812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc067cb0 con 0x7fdefc15e580 2024-03-08T06:04:03.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.238+0000 7fdef3e00640 1 -- 172.21.15.40:0/2740539812 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdef4046020 con 0x7fdefc15e580 2024-03-08T06:04:03.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.239+0000 7fdef3e00640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdedc06b4d0 0x7fdedc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.239+0000 7fdef3e00640 1 -- 172.21.15.40:0/2740539812 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdef40acaa0 con 0x7fdefc15e580 2024-03-08T06:04:03.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.239+0000 7fdefbe00640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdedc06b4d0 0x7fdedc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.240+0000 7fdefbe00640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdedc06b4d0 0x7fdedc06d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdeec007fd0 tx=0x7fdeec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:03.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.244+0000 7fdef3e00640 1 -- 172.21.15.40:0/2740539812 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdef404d030 con 0x7fdefc15e580 2024-03-08T06:04:03.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.442+0000 7fdf03200640 1 -- 172.21.15.40:0/2740539812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdefc06ea30 con 0x7fdefc15e580 2024-03-08T06:04:03.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.445+0000 7fdef3e00640 1 -- 172.21.15.40:0/2740539812 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16534 (secure 0 0 0) 0x7fdef407b9a0 con 0x7fdefc15e580 2024-03-08T06:04:03.445 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:03.445 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:02.002825+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.447+0000 7fdf03200640 1 -- 172.21.15.40:0/2740539812 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdedc06b4d0 msgr2=0x7fdedc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.447+0000 7fdf03200640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdedc06b4d0 0x7fdedc06d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdeec007fd0 tx=0x7fdeec009040 comp rx=0 tx=0).stop 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.447+0000 7fdf03200640 1 -- 172.21.15.40:0/2740539812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdefc15e580 msgr2=0x7fdefc142640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.447+0000 7fdf03200640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdefc15e580 0x7fdefc142640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009f40 tx=0x7fdef4004a40 comp rx=0 tx=0).stop 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.447+0000 7fdf03200640 1 -- 172.21.15.40:0/2740539812 shutdown_connections 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.447+0000 7fdf03200640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdedc06b4d0 0x7fdedc06d990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.447+0000 7fdf03200640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdefc15e580 0x7fdefc142640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.447+0000 7fdf03200640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdefc144030 0x7fdefc142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.447+0000 7fdf03200640 1 --2- 172.21.15.40:0/2740539812 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdefc06ec50 0x7fdefc15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.447+0000 7fdf03200640 1 -- 172.21.15.40:0/2740539812 >> 172.21.15.40:0/2740539812 conn(0x7fdefc06ae30 msgr2=0x7fdefc0ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.448+0000 7fdf03200640 1 -- 172.21.15.40:0/2740539812 shutdown_connections 2024-03-08T06:04:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.448+0000 7fdf03200640 1 -- 172.21.15.40:0/2740539812 wait complete. 2024-03-08T06:04:03.466 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-08T06:04:03.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.688+0000 7f3057200640 1 Processor -- start 2024-03-08T06:04:03.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.688+0000 7f3057200640 1 -- start start 2024-03-08T06:04:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.690+0000 7f3057200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3050145500 0x7f305006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.690+0000 7f3057200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3050071df0 0x7f30500cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.690+0000 7f3057200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305015c460 con 0x7f305015bfb0 2024-03-08T06:04:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.690+0000 7f3057200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305015c740 con 0x7f3050145500 2024-03-08T06:04:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.690+0000 7f3057200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305015ca20 con 0x7f3050071df0 2024-03-08T06:04:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.690+0000 7f3055e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3050145500 0x7f305006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.690+0000 7f3055e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3050145500 0x7f305006ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59826/0 (socket says 172.21.15.40:59826) 2024-03-08T06:04:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.690+0000 7f3055e00640 1 -- 172.21.15.40:0/2468927687 learned_addr learned my addr 172.21.15.40:0/2468927687 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.691+0000 7f3056800640 1 --2- 172.21.15.40:0/2468927687 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3050071df0 0x7f30500cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.691+0000 7f3055e00640 1 -- 172.21.15.40:0/2468927687 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3050071df0 msgr2=0x7f30500cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:03.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.691+0000 7f3055e00640 1 --2- 172.21.15.40:0/2468927687 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3050071df0 0x7f30500cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.691+0000 7f3055e00640 1 -- 172.21.15.40:0/2468927687 >> v1:172.21.15.53:6789/0 conn(0x7f305015bfb0 legacy=0x7f30500716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:03.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.691+0000 7f3055e00640 1 -- 172.21.15.40:0/2468927687 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305015cd00 con 0x7f3050145500 2024-03-08T06:04:03.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.692+0000 7f3055e00640 1 --2- 172.21.15.40:0/2468927687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3050145500 0x7f305006ecb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3044009870 tx=0x7f304402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68c6252dae8518e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:03.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.693+0000 7f3054a00640 1 -- 172.21.15.40:0/2468927687 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044035020 con 0x7f3050145500 2024-03-08T06:04:03.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.693+0000 7f3054a00640 1 -- 172.21.15.40:0/2468927687 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3044031640 con 0x7f3050145500 2024-03-08T06:04:03.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.693+0000 7f3054a00640 1 -- 172.21.15.40:0/2468927687 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304402d040 con 0x7f3050145500 2024-03-08T06:04:03.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.693+0000 7f3057200640 1 -- 172.21.15.40:0/2468927687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3050145500 msgr2=0x7f305006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:03.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.693+0000 7f3057200640 1 --2- 172.21.15.40:0/2468927687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3050145500 0x7f305006ecb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3044009870 tx=0x7f304402b8f0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.694+0000 7f3057200640 1 -- 172.21.15.40:0/2468927687 shutdown_connections 2024-03-08T06:04:03.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.694+0000 7f3057200640 1 --2- 172.21.15.40:0/2468927687 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3050071df0 0x7f30500cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.694+0000 7f3057200640 1 --2- 172.21.15.40:0/2468927687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3050145500 0x7f305006ecb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.694+0000 7f3057200640 1 -- 172.21.15.40:0/2468927687 >> 172.21.15.40:0/2468927687 conn(0x7f30500b1850 msgr2=0x7f30500b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:03.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.694+0000 7f3057200640 1 -- 172.21.15.40:0/2468927687 shutdown_connections 2024-03-08T06:04:03.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.694+0000 7f3057200640 1 -- 172.21.15.40:0/2468927687 wait complete. 2024-03-08T06:04:03.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.695+0000 7f3057200640 1 Processor -- start 2024-03-08T06:04:03.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.695+0000 7f3057200640 1 -- start start 2024-03-08T06:04:03.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.696+0000 7f3057200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3050071df0 0x7f3050142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.696+0000 7f3057200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3050145500 0x7f3050140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.696+0000 7f3057200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f305015bfb0 0x7f305013f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.696+0000 7f3057200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30500ccbc0 con 0x7f3050145500 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.696+0000 7f3057200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30500cca40 con 0x7f3050071df0 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.696+0000 7f3057200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30500ccd40 con 0x7f305015bfb0 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.697+0000 7f3055400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3050145500 0x7f3050140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.697+0000 7f3055400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3050145500 0x7f3050140dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59838/0 (socket says 172.21.15.40:59838) 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.697+0000 7f3055400640 1 -- 172.21.15.40:0/215209238 learned_addr learned my addr 172.21.15.40:0/215209238 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.697+0000 7f3056800640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f305015bfb0 0x7f305013f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.697+0000 7f3055e00640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3050071df0 0x7f3050142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.697+0000 7f3055400640 1 -- 172.21.15.40:0/215209238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f305015bfb0 msgr2=0x7f305013f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.697+0000 7f3055400640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f305015bfb0 0x7f305013f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.697+0000 7f3055400640 1 -- 172.21.15.40:0/215209238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3050071df0 msgr2=0x7f3050142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.697+0000 7f3055400640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3050071df0 0x7f3050142b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.697+0000 7f3055400640 1 -- 172.21.15.40:0/215209238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3050141300 con 0x7f3050145500 2024-03-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.698+0000 7f3055400640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3050145500 0x7f3050140dc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f304000c3c0 tx=0x7f304000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.698+0000 7f304a000640 1 -- 172.21.15.40:0/215209238 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30400052d0 con 0x7f3050145500 2024-03-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.698+0000 7f304a000640 1 -- 172.21.15.40:0/215209238 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3040005490 con 0x7f3050145500 2024-03-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.698+0000 7f3057200640 1 -- 172.21.15.40:0/215209238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3050143970 con 0x7f3050145500 2024-03-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.698+0000 7f304a000640 1 -- 172.21.15.40:0/215209238 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040007260 con 0x7f3050145500 2024-03-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.698+0000 7f3057200640 1 -- 172.21.15.40:0/215209238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3050142300 con 0x7f3050145500 2024-03-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.699+0000 7f3057200640 1 -- 172.21.15.40:0/215209238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3050143e90 con 0x7f3050145500 2024-03-08T06:04:03.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.700+0000 7f304a000640 1 -- 172.21.15.40:0/215209238 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3040007400 con 0x7f3050145500 2024-03-08T06:04:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.701+0000 7f304a000640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f303406b4d0 0x7f303406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:03.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.702+0000 7f304a000640 1 -- 172.21.15.40:0/215209238 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3040089780 con 0x7f3050145500 2024-03-08T06:04:03.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.702+0000 7f3055e00640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f303406b4d0 0x7f303406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:03.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.702+0000 7f3055e00640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f303406b4d0 0x7f303406d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3044036000 tx=0x7f304403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:03.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.706+0000 7f304a000640 1 -- 172.21.15.40:0/215209238 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3050143e90 con 0x7f3050145500 2024-03-08T06:04:03.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:03.942+0000 7f3057200640 1 -- 172.21.15.40:0/215209238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3050075090 con 0x7f3050145500 2024-03-08T06:04:04.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.541+0000 7f304a000640 1 -- 172.21.15.40:0/215209238 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3040058560 con 0x7f3050145500 2024-03-08T06:04:04.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 -- 172.21.15.40:0/215209238 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f303406b4d0 msgr2=0x7f303406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:04.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f303406b4d0 0x7f303406d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3044036000 tx=0x7f304403c040 comp rx=0 tx=0).stop 2024-03-08T06:04:04.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 -- 172.21.15.40:0/215209238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3050145500 msgr2=0x7f3050140dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:04.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3050145500 0x7f3050140dc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f304000c3c0 tx=0x7f304000c880 comp rx=0 tx=0).stop 2024-03-08T06:04:04.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 -- 172.21.15.40:0/215209238 shutdown_connections 2024-03-08T06:04:04.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f303406b4d0 0x7f303406d990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:04.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f305015bfb0 0x7f305013f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:04.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3050145500 0x7f3050140dc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:04.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 --2- 172.21.15.40:0/215209238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3050071df0 0x7f3050142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:04.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 -- 172.21.15.40:0/215209238 >> 172.21.15.40:0/215209238 conn(0x7f30500b1850 msgr2=0x7f305006d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:04.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 -- 172.21.15.40:0/215209238 shutdown_connections 2024-03-08T06:04:04.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.545+0000 7f3057200640 1 -- 172.21.15.40:0/215209238 wait complete. 2024-03-08T06:04:04.564 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-08T06:04:04.564 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-08T06:04:04.716 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-03-08T06:04:04.715+0000 7f3af9000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-08T06:04:04.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.790+0000 7fdd7d200640 1 Processor -- start 2024-03-08T06:04:04.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.790+0000 7fdd7d200640 1 -- start start 2024-03-08T06:04:04.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.791+0000 7fdd7d200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdd780ccc70 0x7fdd780cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:04.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.791+0000 7fdd7d200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd780b2bc0 0x7fdd780b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:04.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.792+0000 7fdd7d200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd78071990 con 0x7fdd78145590 2024-03-08T06:04:04.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.792+0000 7fdd7d200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd780cd620 con 0x7fdd780ccc70 2024-03-08T06:04:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.792+0000 7fdd7d200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd780cf8b0 con 0x7fdd780b2bc0 2024-03-08T06:04:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.792+0000 7fdd6fe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdd780ccc70 0x7fdd780cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.792+0000 7fdd6fe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdd780ccc70 0x7fdd780cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59842/0 (socket says 172.21.15.40:59842) 2024-03-08T06:04:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.792+0000 7fdd6fe00640 1 -- 172.21.15.40:0/1868380145 learned_addr learned my addr 172.21.15.40:0/1868380145 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.792+0000 7fdd76800640 1 --2- 172.21.15.40:0/1868380145 >> v2:172.21.15.53:3300/0 conn(0x7fdd780b2bc0 0x7fdd780b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.792+0000 7fdd6fe00640 1 -- 172.21.15.40:0/1868380145 >> v2:172.21.15.53:3300/0 conn(0x7fdd780b2bc0 msgr2=0x7fdd780b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.792+0000 7fdd6fe00640 1 --2- 172.21.15.40:0/1868380145 >> v2:172.21.15.53:3300/0 conn(0x7fdd780b2bc0 0x7fdd780b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.792+0000 7fdd6fe00640 1 -- 172.21.15.40:0/1868380145 >> v1:172.21.15.53:6789/0 conn(0x7fdd78145590 legacy=0x7fdd780cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.792+0000 7fdd6fe00640 1 -- 172.21.15.40:0/1868380145 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd78071c70 con 0x7fdd780ccc70 2024-03-08T06:04:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.794+0000 7fdd6fe00640 1 --2- 172.21.15.40:0/1868380145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780ccc70 0x7fdd780cd0e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdd78153230 tx=0x7fdd6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf287532f5b36f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.794+0000 7fdd75400640 1 -- 172.21.15.40:0/1868380145 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60035020 con 0x7fdd780ccc70 2024-03-08T06:04:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.794+0000 7fdd75400640 1 -- 172.21.15.40:0/1868380145 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd60031870 con 0x7fdd780ccc70 2024-03-08T06:04:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.794+0000 7fdd75400640 1 -- 172.21.15.40:0/1868380145 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60031c40 con 0x7fdd780ccc70 2024-03-08T06:04:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.795+0000 7fdd7d200640 1 -- 172.21.15.40:0/1868380145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780ccc70 msgr2=0x7fdd780cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.795+0000 7fdd7d200640 1 --2- 172.21.15.40:0/1868380145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780ccc70 0x7fdd780cd0e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdd78153230 tx=0x7fdd6002b910 comp rx=0 tx=0).stop 2024-03-08T06:04:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.795+0000 7fdd7d200640 1 -- 172.21.15.40:0/1868380145 shutdown_connections 2024-03-08T06:04:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.795+0000 7fdd7d200640 1 --2- 172.21.15.40:0/1868380145 >> v2:172.21.15.53:3300/0 conn(0x7fdd780b2bc0 0x7fdd780b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.795+0000 7fdd7d200640 1 --2- 172.21.15.40:0/1868380145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780ccc70 0x7fdd780cd0e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.795+0000 7fdd7d200640 1 -- 172.21.15.40:0/1868380145 >> 172.21.15.40:0/1868380145 conn(0x7fdd780cfaa0 msgr2=0x7fdd780cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.795+0000 7fdd7d200640 1 -- 172.21.15.40:0/1868380145 shutdown_connections 2024-03-08T06:04:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.795+0000 7fdd7d200640 1 -- 172.21.15.40:0/1868380145 wait complete. 2024-03-08T06:04:04.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.797+0000 7fdd7d200640 1 Processor -- start 2024-03-08T06:04:04.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.797+0000 7fdd7d200640 1 -- start start 2024-03-08T06:04:04.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd7d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd780b2bc0 0x7fdd7815ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd7d200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780ccc70 0x7fdd781f7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd7d200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdd78145590 0x7fdd7813c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd7d200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78072510 con 0x7fdd780ccc70 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd7d200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78072230 con 0x7fdd780b2bc0 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd7d200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78145970 con 0x7fdd78145590 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd6fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780ccc70 0x7fdd781f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd6fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780ccc70 0x7fdd781f7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59852/0 (socket says 172.21.15.40:59852) 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd6fe00640 1 -- 172.21.15.40:0/2006431041 learned_addr learned my addr 172.21.15.40:0/2006431041 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd76800640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdd78145590 0x7fdd7813c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd75e00640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd780b2bc0 0x7fdd7815ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd6fe00640 1 -- 172.21.15.40:0/2006431041 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdd78145590 msgr2=0x7fdd7813c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd6fe00640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdd78145590 0x7fdd7813c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd6fe00640 1 -- 172.21.15.40:0/2006431041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd780b2bc0 msgr2=0x7fdd7815ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd6fe00640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd780b2bc0 0x7fdd7815ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.798+0000 7fdd6fe00640 1 -- 172.21.15.40:0/2006431041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd781f7670 con 0x7fdd780ccc70 2024-03-08T06:04:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.799+0000 7fdd6fe00640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780ccc70 0x7fdd781f7130 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd60000c00 tx=0x7fdd600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.799+0000 7fdd6e000640 1 -- 172.21.15.40:0/2006431041 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60040070 con 0x7fdd780ccc70 2024-03-08T06:04:04.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.799+0000 7fdd6e000640 1 -- 172.21.15.40:0/2006431041 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd60030d00 con 0x7fdd780ccc70 2024-03-08T06:04:04.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.799+0000 7fdd6e000640 1 -- 172.21.15.40:0/2006431041 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60035040 con 0x7fdd780ccc70 2024-03-08T06:04:04.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.799+0000 7fdd7d200640 1 -- 172.21.15.40:0/2006431041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd781f7940 con 0x7fdd780ccc70 2024-03-08T06:04:04.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.799+0000 7fdd7d200640 1 -- 172.21.15.40:0/2006431041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd781f7f00 con 0x7fdd780ccc70 2024-03-08T06:04:04.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.800+0000 7fdd7d200640 1 -- 172.21.15.40:0/2006431041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd78067cb0 con 0x7fdd780ccc70 2024-03-08T06:04:04.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.801+0000 7fdd6e000640 1 -- 172.21.15.40:0/2006431041 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd6004b060 con 0x7fdd780ccc70 2024-03-08T06:04:04.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.802+0000 7fdd6e000640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdd5406b4d0 0x7fdd5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:04.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.802+0000 7fdd6e000640 1 -- 172.21.15.40:0/2006431041 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd600ac060 con 0x7fdd780ccc70 2024-03-08T06:04:04.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.803+0000 7fdd75e00640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdd5406b4d0 0x7fdd5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:04.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.803+0000 7fdd75e00640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdd5406b4d0 0x7fdd5406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd68003d90 tx=0x7fdd68009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:04.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:04.807+0000 7fdd6e000640 1 -- 172.21.15.40:0/2006431041 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd6007ae40 con 0x7fdd780ccc70 2024-03-08T06:04:05.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.044+0000 7fdd7d200640 1 -- 172.21.15.40:0/2006431041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdd780aeb50 con 0x7fdd780ccc70 2024-03-08T06:04:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.545+0000 7fdd6e000640 1 -- 172.21.15.40:0/2006431041 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdd6007a740 con 0x7fdd780ccc70 2024-03-08T06:04:05.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.548+0000 7fdd7d200640 1 -- 172.21.15.40:0/2006431041 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdd5406b4d0 msgr2=0x7fdd5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:05.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.548+0000 7fdd7d200640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdd5406b4d0 0x7fdd5406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd68003d90 tx=0x7fdd68009040 comp rx=0 tx=0).stop 2024-03-08T06:04:05.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.548+0000 7fdd7d200640 1 -- 172.21.15.40:0/2006431041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780ccc70 msgr2=0x7fdd781f7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:05.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.548+0000 7fdd7d200640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780ccc70 0x7fdd781f7130 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd60000c00 tx=0x7fdd600046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:05.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.548+0000 7fdd7d200640 1 -- 172.21.15.40:0/2006431041 shutdown_connections 2024-03-08T06:04:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.548+0000 7fdd7d200640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdd5406b4d0 0x7fdd5406d990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.548+0000 7fdd7d200640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdd78145590 0x7fdd7813c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.548+0000 7fdd7d200640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780ccc70 0x7fdd781f7130 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.548+0000 7fdd7d200640 1 --2- 172.21.15.40:0/2006431041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd780b2bc0 0x7fdd7815ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.548+0000 7fdd7d200640 1 -- 172.21.15.40:0/2006431041 >> 172.21.15.40:0/2006431041 conn(0x7fdd780cfaa0 msgr2=0x7fdd780b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.549+0000 7fdd7d200640 1 -- 172.21.15.40:0/2006431041 shutdown_connections 2024-03-08T06:04:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.549+0000 7fdd7d200640 1 -- 172.21.15.40:0/2006431041 wait complete. 2024-03-08T06:04:05.572 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-08T06:04:05.573 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:04:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.797+0000 7fa12ea00640 1 Processor -- start 2024-03-08T06:04:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.797+0000 7fa12ea00640 1 -- start start 2024-03-08T06:04:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.799+0000 7fa12ea00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa12815bea0 0x7fa12815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.799+0000 7fa12ea00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12815ceb0 con 0x7fa128145520 2024-03-08T06:04:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.799+0000 7fa12ea00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12815c330 con 0x7fa128071df0 2024-03-08T06:04:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.799+0000 7fa12ea00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12815c610 con 0x7fa12815bea0 2024-03-08T06:04:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.799+0000 7fa12d600640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa128145520 0x7fa1280716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42010/0 (socket says 172.21.15.40:42010) 2024-03-08T06:04:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.799+0000 7fa12d600640 1 -- 172.21.15.40:0/3501508570 learned_addr learned my addr 172.21.15.40:0/3501508570 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.799+0000 7fa12cc00640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa12815bea0 0x7fa12815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.799+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540069625 0 0) 0x7fa12815ceb0 con 0x7fa128145520 2024-03-08T06:04:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.799+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa10c003520 con 0x7fa128145520 2024-03-08T06:04:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1892700650 0 0) 0x7fa10c003520 con 0x7fa128145520 2024-03-08T06:04:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa12815bea0 msgr2=0x7fa12815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa12815bea0 0x7fa12815d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 >> v1:172.21.15.53:6789/0 conn(0x7fa128071df0 legacy=0x7fa12806d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12815c8f0 con 0x7fa128145520 2024-03-08T06:04:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fa118001670 con 0x7fa128145520 2024-03-08T06:04:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 >> v1:172.21.15.40:6789/0 conn(0x7fa128145520 legacy=0x7fa1280716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa10c0039e0 0x7fa10c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c004390 0x7fa10c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 0x7fa10c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12815c610 con 0x7fa10c005060 2024-03-08T06:04:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c00bf00 con 0x7fa10c004390 2024-03-08T06:04:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c00c0d0 con 0x7fa10c0039e0 2024-03-08T06:04:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.800+0000 7fa12e000640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 0x7fa10c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.801+0000 7fa12cc00640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c004390 0x7fa10c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.801+0000 7fa12d600640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa10c0039e0 0x7fa10c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.801+0000 7fa12e000640 1 -- 172.21.15.40:0/3501508570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa10c0039e0 msgr2=0x7fa10c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.801+0000 7fa12e000640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa10c0039e0 0x7fa10c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.801+0000 7fa12e000640 1 -- 172.21.15.40:0/3501508570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c004390 msgr2=0x7fa10c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.801+0000 7fa12e000640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c004390 0x7fa10c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.801+0000 7fa12e000640 1 -- 172.21.15.40:0/3501508570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c003520 con 0x7fa10c005060 2024-03-08T06:04:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.802+0000 7fa12e000640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 0x7fa10c00b9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa120001e50 tx=0x7fa10c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78854a4d8b8ee6c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.802+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120001a70 con 0x7fa10c005060 2024-03-08T06:04:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.802+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa120003b70 con 0x7fa10c005060 2024-03-08T06:04:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.802+0000 7fa127e00640 1 -- 172.21.15.40:0/3501508570 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1200050d0 con 0x7fa10c005060 2024-03-08T06:04:05.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.803+0000 7fa12ea00640 1 -- 172.21.15.40:0/3501508570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 msgr2=0x7fa10c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:05.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.803+0000 7fa12ea00640 1 --2- 172.21.15.40:0/3501508570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 0x7fa10c00b9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa120001e50 tx=0x7fa10c005830 comp rx=0 tx=0).stop 2024-03-08T06:04:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.803+0000 7fa12e000640 1 -- 172.21.15.40:0/3501508570 reap_dead start 2024-03-08T06:04:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.803+0000 7fa12ea00640 1 -- 172.21.15.40:0/3501508570 shutdown_connections 2024-03-08T06:04:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.803+0000 7fa12ea00640 1 -- 172.21.15.40:0/3501508570 >> 172.21.15.40:0/3501508570 conn(0x7fa1280b1850 msgr2=0x7fa1280b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.803+0000 7fa12ea00640 1 -- 172.21.15.40:0/3501508570 shutdown_connections 2024-03-08T06:04:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.803+0000 7fa12ea00640 1 -- 172.21.15.40:0/3501508570 wait complete. 2024-03-08T06:04:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.805+0000 7fa12ea00640 1 Processor -- start 2024-03-08T06:04:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.805+0000 7fa12ea00640 1 -- start start 2024-03-08T06:04:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 0x7fa1280b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12815d7d0 0x7fa128141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12ea00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa1281594b0 0x7fa128141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12ea00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280af1c0 con 0x7fa10c005060 2024-03-08T06:04:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12ea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280af040 con 0x7fa12815d7d0 2024-03-08T06:04:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12ea00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12806e410 con 0x7fa1281594b0 2024-03-08T06:04:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12d600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 0x7fa1280b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12d600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 0x7fa1280b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59868/0 (socket says 172.21.15.40:59868) 2024-03-08T06:04:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12d600640 1 -- 172.21.15.40:0/4047071424 learned_addr learned my addr 172.21.15.40:0/4047071424 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12e000640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa1281594b0 0x7fa128141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12cc00640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12815d7d0 0x7fa128141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12d600640 1 -- 172.21.15.40:0/4047071424 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa1281594b0 msgr2=0x7fa128141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.806+0000 7fa12d600640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa1281594b0 0x7fa128141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.807+0000 7fa12d600640 1 -- 172.21.15.40:0/4047071424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12815d7d0 msgr2=0x7fa128141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.807+0000 7fa12d600640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12815d7d0 0x7fa128141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.807+0000 7fa12d600640 1 -- 172.21.15.40:0/4047071424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1280b1210 con 0x7fa10c005060 2024-03-08T06:04:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.807+0000 7fa12e000640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa1281594b0 0x7fa128141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.807+0000 7fa12cc00640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12815d7d0 0x7fa128141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.807+0000 7fa12d600640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 0x7fa1280b08a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa11800e860 tx=0x7fa11800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.807+0000 7fa125600640 1 -- 172.21.15.40:0/4047071424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800c870 con 0x7fa10c005060 2024-03-08T06:04:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.807+0000 7fa125600640 1 -- 172.21.15.40:0/4047071424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa118018070 con 0x7fa10c005060 2024-03-08T06:04:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.807+0000 7fa125600640 1 -- 172.21.15.40:0/4047071424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800cc70 con 0x7fa10c005060 2024-03-08T06:04:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.807+0000 7fa12ea00640 1 -- 172.21.15.40:0/4047071424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1280af690 con 0x7fa10c005060 2024-03-08T06:04:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.807+0000 7fa12ea00640 1 -- 172.21.15.40:0/4047071424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1280b0de0 con 0x7fa10c005060 2024-03-08T06:04:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.808+0000 7fa12ea00640 1 -- 172.21.15.40:0/4047071424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa128075100 con 0x7fa10c005060 2024-03-08T06:04:05.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.809+0000 7fa125600640 1 -- 172.21.15.40:0/4047071424 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa118013420 con 0x7fa10c005060 2024-03-08T06:04:05.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.810+0000 7fa125600640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa10806b500 0x7fa10806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:05.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.810+0000 7fa125600640 1 -- 172.21.15.40:0/4047071424 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa118089fc0 con 0x7fa10c005060 2024-03-08T06:04:05.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.810+0000 7fa12cc00640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa10806b500 0x7fa10806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:05.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.811+0000 7fa12cc00640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa10806b500 0x7fa10806d9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa1280b4a60 tx=0x7fa114008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:05.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:05.815+0000 7fa125600640 1 -- 172.21.15.40:0/4047071424 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa118058da0 con 0x7fa10c005060 2024-03-08T06:04:06.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.055+0000 7fa12ea00640 1 -- 172.21.15.40:0/4047071424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1281599b0 con 0x7fa10c005060 2024-03-08T06:04:06.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.056+0000 7fa125600640 1 -- 172.21.15.40:0/4047071424 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fa1180586a0 con 0x7fa10c005060 2024-03-08T06:04:06.056 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:06.057 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3266388900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3266388900},{"type":"v1","addr":"172.21.15.40:6837","nonce":3266388900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2471668569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2471668569},{"type":"v1","addr":"172.21.15.53:6834","nonce":2471668569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3262228631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3262228631},{"type":"v1","addr":"172.21.15.53:6835","nonce":3262228631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:04:02.003553+0000","modified":"2024-03-08T06:04:05.045865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3709744112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3709744112},{"type":"v1","addr":"172.21.15.40:6835","nonce":3709744112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:04:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-03-08T06:04:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.058+0000 7fa12ea00640 1 -- 172.21.15.40:0/4047071424 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa10806b500 msgr2=0x7fa10806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.058+0000 7fa12ea00640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa10806b500 0x7fa10806d9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa1280b4a60 tx=0x7fa114008040 comp rx=0 tx=0).stop 2024-03-08T06:04:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.058+0000 7fa12ea00640 1 -- 172.21.15.40:0/4047071424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 msgr2=0x7fa1280b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.058+0000 7fa12ea00640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 0x7fa1280b08a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa11800e860 tx=0x7fa11800ed20 comp rx=0 tx=0).stop 2024-03-08T06:04:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.059+0000 7fa12ea00640 1 -- 172.21.15.40:0/4047071424 shutdown_connections 2024-03-08T06:04:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.059+0000 7fa12ea00640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa10806b500 0x7fa10806d9c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.059+0000 7fa12ea00640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa1281594b0 0x7fa128141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.059+0000 7fa12ea00640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12815d7d0 0x7fa128141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.059+0000 7fa12ea00640 1 --2- 172.21.15.40:0/4047071424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa10c005060 0x7fa1280b08a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.059+0000 7fa12ea00640 1 -- 172.21.15.40:0/4047071424 >> 172.21.15.40:0/4047071424 conn(0x7fa1280b1850 msgr2=0x7fa12815b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.059+0000 7fa12ea00640 1 -- 172.21.15.40:0/4047071424 shutdown_connections 2024-03-08T06:04:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.059+0000 7fa12ea00640 1 -- 172.21.15.40:0/4047071424 wait complete. 2024-03-08T06:04:06.079 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:06.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.297+0000 7f9241200640 1 Processor -- start 2024-03-08T06:04:06.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.298+0000 7f9241200640 1 -- start start 2024-03-08T06:04:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.299+0000 7f9241200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f923c15df20 0x7f923c0ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.299+0000 7f9241200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f923c0d11b0 0x7f923c162400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.299+0000 7f9241200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923c15e300 con 0x7f923c0b6bd0 2024-03-08T06:04:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.299+0000 7f9241200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923c0d3990 con 0x7f923c15df20 2024-03-08T06:04:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.299+0000 7f9241200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c162940 con 0x7f923c0d11b0 2024-03-08T06:04:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f9232a00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f923c0b6bd0 0x7f923c0d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42018/0 (socket says 172.21.15.40:42018) 2024-03-08T06:04:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f9232a00640 1 -- 172.21.15.40:0/3739816037 learned_addr learned my addr 172.21.15.40:0/3739816037 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f923b400640 1 --2- 172.21.15.40:0/3739816037 >> v2:172.21.15.53:3300/0 conn(0x7f923c15df20 0x7f923c0ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f923be00640 1 --2- 172.21.15.40:0/3739816037 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f923c0d11b0 0x7f923c162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204744415 0 0) 0x7f923c15e300 con 0x7f923c0b6bd0 2024-03-08T06:04:06.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9220003520 con 0x7f923c0b6bd0 2024-03-08T06:04:06.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 497072313 0 0) 0x7f9220003520 con 0x7f923c0b6bd0 2024-03-08T06:04:06.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f923c0d11b0 msgr2=0x7f923c162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:06.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f923aa00640 1 --2- 172.21.15.40:0/3739816037 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f923c0d11b0 0x7f923c162400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 >> v2:172.21.15.53:3300/0 conn(0x7f923c15df20 msgr2=0x7f923c0ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f923aa00640 1 --2- 172.21.15.40:0/3739816037 >> v2:172.21.15.53:3300/0 conn(0x7f923c15df20 0x7f923c0ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c0cf030 con 0x7f923c0b6bd0 2024-03-08T06:04:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.300+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f9224001670 con 0x7f923c0b6bd0 2024-03-08T06:04:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 >> v1:172.21.15.40:6789/0 conn(0x7f923c0b6bd0 legacy=0x7f923c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923aa00640 1 --2- 172.21.15.40:0/3739816037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 0x7f9220003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923aa00640 1 --2- 172.21.15.40:0/3739816037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9220004800 0x7f9220004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923aa00640 1 --2- 172.21.15.40:0/3739816037 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f92200051c0 0x7f922000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0d3990 con 0x7f92200039b0 2024-03-08T06:04:06.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c162940 con 0x7f9220004800 2024-03-08T06:04:06.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922000bf80 con 0x7f92200051c0 2024-03-08T06:04:06.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923be00640 1 --2- 172.21.15.40:0/3739816037 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f923c0d11b0 0x7f923c162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:06.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923b400640 1 --2- 172.21.15.40:0/3739816037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 0x7f9220003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:06.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923b400640 1 -- 172.21.15.40:0/3739816037 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f92200051c0 msgr2=0x7f922000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:06.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923b400640 1 --2- 172.21.15.40:0/3739816037 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f92200051c0 0x7f922000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923b400640 1 -- 172.21.15.40:0/3739816037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9220004800 msgr2=0x7f9220004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:06.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923b400640 1 --2- 172.21.15.40:0/3739816037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9220004800 0x7f9220004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.301+0000 7f923b400640 1 -- 172.21.15.40:0/3739816037 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9220003520 con 0x7f92200039b0 2024-03-08T06:04:06.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.302+0000 7f923b400640 1 --2- 172.21.15.40:0/3739816037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 0x7f9220003dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f922c009ac0 tx=0x7f9220005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf76679cef68934 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:06.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.303+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c039070 con 0x7f92200039b0 2024-03-08T06:04:06.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.303+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922c02c710 con 0x7f92200039b0 2024-03-08T06:04:06.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.303+0000 7f923aa00640 1 -- 172.21.15.40:0/3739816037 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c009690 con 0x7f92200039b0 2024-03-08T06:04:06.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.303+0000 7f9241200640 1 -- 172.21.15.40:0/3739816037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 msgr2=0x7f9220003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.303+0000 7f9241200640 1 --2- 172.21.15.40:0/3739816037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 0x7f9220003dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f922c009ac0 tx=0x7f9220005830 comp rx=0 tx=0).stop 2024-03-08T06:04:06.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.303+0000 7f923be00640 1 -- 172.21.15.40:0/3739816037 reap_dead start 2024-03-08T06:04:06.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.303+0000 7f9241200640 1 -- 172.21.15.40:0/3739816037 shutdown_connections 2024-03-08T06:04:06.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.303+0000 7f9241200640 1 -- 172.21.15.40:0/3739816037 >> 172.21.15.40:0/3739816037 conn(0x7f923c06c740 msgr2=0x7f923c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:06.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.303+0000 7f9241200640 1 -- 172.21.15.40:0/3739816037 shutdown_connections 2024-03-08T06:04:06.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.303+0000 7f9241200640 1 -- 172.21.15.40:0/3739816037 wait complete. 2024-03-08T06:04:06.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.305+0000 7f9241200640 1 Processor -- start 2024-03-08T06:04:06.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.305+0000 7f9241200640 1 -- start start 2024-03-08T06:04:06.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.306+0000 7f9241200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 0x7f923c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.306+0000 7f9241200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f923c0cb0a0 0x7f923c15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.306+0000 7f9241200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f923c0b8770 0x7f923c154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.306+0000 7f9241200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0d2180 con 0x7f92200039b0 2024-03-08T06:04:06.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.306+0000 7f9241200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0d1ea0 con 0x7f923c0cb0a0 2024-03-08T06:04:06.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.306+0000 7f9241200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0d2460 con 0x7f923c0b8770 2024-03-08T06:04:06.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.306+0000 7f923b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 0x7f923c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:06.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.306+0000 7f923b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 0x7f923c1604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59882/0 (socket says 172.21.15.40:59882) 2024-03-08T06:04:06.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.306+0000 7f923b400640 1 -- 172.21.15.40:0/3361061062 learned_addr learned my addr 172.21.15.40:0/3361061062 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:06.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f923be00640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f923c0b8770 0x7f923c154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:06.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f923b400640 1 -- 172.21.15.40:0/3361061062 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f923c0b8770 msgr2=0x7f923c154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f923b400640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f923c0b8770 0x7f923c154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f923b400640 1 -- 172.21.15.40:0/3361061062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f923c0cb0a0 msgr2=0x7f923c15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f9232a00640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f923c0cb0a0 0x7f923c15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:06.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f923b400640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f923c0cb0a0 0x7f923c15e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f923b400640 1 -- 172.21.15.40:0/3361061062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c0cb520 con 0x7f92200039b0 2024-03-08T06:04:06.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f923b400640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 0x7f923c1604b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f922c02c380 tx=0x7f922c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:06.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f9233e00640 1 -- 172.21.15.40:0/3361061062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c043020 con 0x7f92200039b0 2024-03-08T06:04:06.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f9233e00640 1 -- 172.21.15.40:0/3361061062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922c0044c0 con 0x7f92200039b0 2024-03-08T06:04:06.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f9233e00640 1 -- 172.21.15.40:0/3361061062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c039040 con 0x7f92200039b0 2024-03-08T06:04:06.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f9241200640 1 -- 172.21.15.40:0/3361061062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923c15ec40 con 0x7f92200039b0 2024-03-08T06:04:06.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.307+0000 7f9241200640 1 -- 172.21.15.40:0/3361061062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c15f200 con 0x7f92200039b0 2024-03-08T06:04:06.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.308+0000 7f9241200640 1 -- 172.21.15.40:0/3361061062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923c075100 con 0x7f92200039b0 2024-03-08T06:04:06.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.309+0000 7f9233e00640 1 -- 172.21.15.40:0/3361061062 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f922c046020 con 0x7f92200039b0 2024-03-08T06:04:06.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.310+0000 7f9233e00640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f921806b4d0 0x7f921806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.310+0000 7f9233e00640 1 -- 172.21.15.40:0/3361061062 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f922c0abb50 con 0x7f92200039b0 2024-03-08T06:04:06.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.311+0000 7f9232a00640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f921806b4d0 0x7f921806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:06.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.311+0000 7f9232a00640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f921806b4d0 0x7f921806d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9224001eb0 tx=0x7f9224009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:06.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.315+0000 7f9233e00640 1 -- 172.21.15.40:0/3361061062 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f922c04d070 con 0x7f92200039b0 2024-03-08T06:04:06.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.507+0000 7f9241200640 1 -- 172.21.15.40:0/3361061062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f923c161520 con 0x7f92200039b0 2024-03-08T06:04:06.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.510+0000 7f9233e00640 1 -- 172.21.15.40:0/3361061062 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f923c0cb520 con 0x7f92200039b0 2024-03-08T06:04:06.511 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:06.511 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:06.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 -- 172.21.15.40:0/3361061062 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f921806b4d0 msgr2=0x7f921806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f921806b4d0 0x7f921806d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9224001eb0 tx=0x7f9224009040 comp rx=0 tx=0).stop 2024-03-08T06:04:06.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 -- 172.21.15.40:0/3361061062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 msgr2=0x7f923c1604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 0x7f923c1604b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f922c02c380 tx=0x7f922c004730 comp rx=0 tx=0).stop 2024-03-08T06:04:06.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 -- 172.21.15.40:0/3361061062 shutdown_connections 2024-03-08T06:04:06.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f921806b4d0 0x7f921806d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f923c0b8770 0x7f923c154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f923c0cb0a0 0x7f923c15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 --2- 172.21.15.40:0/3361061062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92200039b0 0x7f923c1604b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 -- 172.21.15.40:0/3361061062 >> 172.21.15.40:0/3361061062 conn(0x7f923c06c740 msgr2=0x7f923c0b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:06.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 -- 172.21.15.40:0/3361061062 shutdown_connections 2024-03-08T06:04:06.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.513+0000 7f9241200640 1 -- 172.21.15.40:0/3361061062 wait complete. 2024-03-08T06:04:06.532 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-08T06:04:06.532 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:04:06.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.751+0000 7fcb64a00640 1 Processor -- start 2024-03-08T06:04:06.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.751+0000 7fcb64a00640 1 -- start start 2024-03-08T06:04:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.753+0000 7fcb64a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcb6015bea0 0x7fcb6015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.753+0000 7fcb64a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb60071df0 0x7fcb6006d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.753+0000 7fcb64a00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6015ceb0 con 0x7fcb60145520 2024-03-08T06:04:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.753+0000 7fcb64a00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6015c330 con 0x7fcb60071df0 2024-03-08T06:04:06.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.753+0000 7fcb64a00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6015c610 con 0x7fcb6015bea0 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.753+0000 7fcb5f200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb60071df0 0x7fcb6006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.753+0000 7fcb5f200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb60071df0 0x7fcb6006d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59892/0 (socket says 172.21.15.40:59892) 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.753+0000 7fcb5f200640 1 -- 172.21.15.40:0/3500462839 learned_addr learned my addr 172.21.15.40:0/3500462839 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.754+0000 7fcb5de00640 1 --2- 172.21.15.40:0/3500462839 >> v2:172.21.15.53:3300/0 conn(0x7fcb6015bea0 0x7fcb6015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.754+0000 7fcb5f200640 1 -- 172.21.15.40:0/3500462839 >> v2:172.21.15.53:3300/0 conn(0x7fcb6015bea0 msgr2=0x7fcb6015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.754+0000 7fcb5f200640 1 --2- 172.21.15.40:0/3500462839 >> v2:172.21.15.53:3300/0 conn(0x7fcb6015bea0 0x7fcb6015d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.754+0000 7fcb5f200640 1 -- 172.21.15.40:0/3500462839 >> v1:172.21.15.53:6789/0 conn(0x7fcb60145520 legacy=0x7fcb600716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.754+0000 7fcb5f200640 1 -- 172.21.15.40:0/3500462839 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6015c8f0 con 0x7fcb60071df0 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.755+0000 7fcb5f200640 1 --2- 172.21.15.40:0/3500462839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb60071df0 0x7fcb6006d570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb6006be10 tx=0x7fcb5802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8113a5965d963f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.755+0000 7fcb5d400640 1 -- 172.21.15.40:0/3500462839 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5802cb20 con 0x7fcb60071df0 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.755+0000 7fcb5d400640 1 -- 172.21.15.40:0/3500462839 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5802ccc0 con 0x7fcb60071df0 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.755+0000 7fcb5d400640 1 -- 172.21.15.40:0/3500462839 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58034610 con 0x7fcb60071df0 2024-03-08T06:04:06.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.756+0000 7fcb64a00640 1 -- 172.21.15.40:0/3500462839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb60071df0 msgr2=0x7fcb6006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.756+0000 7fcb64a00640 1 --2- 172.21.15.40:0/3500462839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb60071df0 0x7fcb6006d570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb6006be10 tx=0x7fcb5802bf80 comp rx=0 tx=0).stop 2024-03-08T06:04:06.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.756+0000 7fcb64a00640 1 -- 172.21.15.40:0/3500462839 shutdown_connections 2024-03-08T06:04:06.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.756+0000 7fcb64a00640 1 --2- 172.21.15.40:0/3500462839 >> v2:172.21.15.53:3300/0 conn(0x7fcb6015bea0 0x7fcb6015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.756+0000 7fcb64a00640 1 --2- 172.21.15.40:0/3500462839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb60071df0 0x7fcb6006d570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.756+0000 7fcb64a00640 1 -- 172.21.15.40:0/3500462839 >> 172.21.15.40:0/3500462839 conn(0x7fcb600b1850 msgr2=0x7fcb600b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:06.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.756+0000 7fcb64a00640 1 -- 172.21.15.40:0/3500462839 shutdown_connections 2024-03-08T06:04:06.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.756+0000 7fcb64a00640 1 -- 172.21.15.40:0/3500462839 wait complete. 2024-03-08T06:04:06.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.757+0000 7fcb64a00640 1 Processor -- start 2024-03-08T06:04:06.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.758+0000 7fcb64a00640 1 -- start start 2024-03-08T06:04:06.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.758+0000 7fcb64a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcb60071df0 0x7fcb600a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.758+0000 7fcb64a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb60145520 0x7fcb601f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.758+0000 7fcb64a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb600a9700 0x7fcb600a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb64a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600af1c0 con 0x7fcb60145520 2024-03-08T06:04:06.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb64a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600af040 con 0x7fcb600a9700 2024-03-08T06:04:06.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb64a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6006e410 con 0x7fcb60071df0 2024-03-08T06:04:06.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb5de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb60145520 0x7fcb601f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:06.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb5de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb60145520 0x7fcb601f26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59908/0 (socket says 172.21.15.40:59908) 2024-03-08T06:04:06.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb5de00640 1 -- 172.21.15.40:0/2586967408 learned_addr learned my addr 172.21.15.40:0/2586967408 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:06.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb5de00640 1 -- 172.21.15.40:0/2586967408 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcb60071df0 msgr2=0x7fcb600a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb5de00640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcb60071df0 0x7fcb600a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb5de00640 1 -- 172.21.15.40:0/2586967408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb600a9700 msgr2=0x7fcb600a9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb5de00640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb600a9700 0x7fcb600a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb5de00640 1 -- 172.21.15.40:0/2586967408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb601423d0 con 0x7fcb60145520 2024-03-08T06:04:06.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.759+0000 7fcb5de00640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb60145520 0x7fcb601f26e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00d620 tx=0x7fcb4c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:06.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.760+0000 7fcb56a00640 1 -- 172.21.15.40:0/2586967408 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c00b620 con 0x7fcb60145520 2024-03-08T06:04:06.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.760+0000 7fcb56a00640 1 -- 172.21.15.40:0/2586967408 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c019070 con 0x7fcb60145520 2024-03-08T06:04:06.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.760+0000 7fcb56a00640 1 -- 172.21.15.40:0/2586967408 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c00ba00 con 0x7fcb60145520 2024-03-08T06:04:06.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.760+0000 7fcb64a00640 1 -- 172.21.15.40:0/2586967408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb601426b0 con 0x7fcb60145520 2024-03-08T06:04:06.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.760+0000 7fcb64a00640 1 -- 172.21.15.40:0/2586967408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb60142c70 con 0x7fcb60145520 2024-03-08T06:04:06.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.761+0000 7fcb64a00640 1 -- 172.21.15.40:0/2586967408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb60075100 con 0x7fcb60145520 2024-03-08T06:04:06.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.762+0000 7fcb56a00640 1 -- 172.21.15.40:0/2586967408 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb4c00bba0 con 0x7fcb60145520 2024-03-08T06:04:06.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.762+0000 7fcb56a00640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcb4006b4d0 0x7fcb4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:06.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.763+0000 7fcb56a00640 1 -- 172.21.15.40:0/2586967408 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb4c08a5e0 con 0x7fcb60145520 2024-03-08T06:04:06.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.763+0000 7fcb5e800640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcb4006b4d0 0x7fcb4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:06.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.764+0000 7fcb5e800640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcb4006b4d0 0x7fcb4006d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000f60 tx=0x7fcb500079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:06.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.767+0000 7fcb56a00640 1 -- 172.21.15.40:0/2586967408 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb4c059450 con 0x7fcb60145520 2024-03-08T06:04:06.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.960+0000 7fcb64a00640 1 -- 172.21.15.40:0/2586967408 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb600ce7b0 con 0x7fcb4006b4d0 2024-03-08T06:04:06.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.972+0000 7fcb56a00640 1 -- 172.21.15.40:0/2586967408 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251936 (secure 0 0 0) 0x7fcb600ce7b0 con 0x7fcb4006b4d0 2024-03-08T06:04:06.976 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:06.977 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-03-08T06:04:06.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 -- 172.21.15.40:0/2586967408 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcb4006b4d0 msgr2=0x7fcb4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcb4006b4d0 0x7fcb4006d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000f60 tx=0x7fcb500079b0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 -- 172.21.15.40:0/2586967408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb60145520 msgr2=0x7fcb601f26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:06.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb60145520 0x7fcb601f26e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00d620 tx=0x7fcb4c00dae0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 -- 172.21.15.40:0/2586967408 shutdown_connections 2024-03-08T06:04:06.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcb4006b4d0 0x7fcb4006d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb600a9700 0x7fcb600a9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb60145520 0x7fcb601f26e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 --2- 172.21.15.40:0/2586967408 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcb60071df0 0x7fcb600a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:06.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 -- 172.21.15.40:0/2586967408 >> 172.21.15.40:0/2586967408 conn(0x7fcb600b1850 msgr2=0x7fcb6013fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:06.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 -- 172.21.15.40:0/2586967408 shutdown_connections 2024-03-08T06:04:06.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:06.977+0000 7fcb64a00640 1 -- 172.21.15.40:0/2586967408 wait complete. 2024-03-08T06:04:06.996 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-03-08T06:04:05.483554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236112,"kb_used_data":2884,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738608,"statfs":{"total":773094113280,"available":772852334592,"internally_reserved":0,"allocated":2953216,"data_stored":1771496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.726509"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618708+0000","last_change":"2024-03-08T06:04:02.552950+0000","last_active":"2024-03-08T06:04:03.618708+0000","last_peered":"2024-03-08T06:04:03.618708+0000","last_clean":"2024-03-08T06:04:03.618708+0000","last_became_active":"2024-03-08T06:04:02.552600+0000","last_became_peered":"2024-03-08T06:04:02.552600+0000","last_unstale":"2024-03-08T06:04:03.618708+0000","last_undegraded":"2024-03-08T06:04:03.618708+0000","last_fullsized":"2024-03-08T06:04:03.618708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:13:36.393811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553892+0000","last_change":"2024-03-08T06:04:02.547475+0000","last_active":"2024-03-08T06:04:03.553892+0000","last_peered":"2024-03-08T06:04:03.553892+0000","last_clean":"2024-03-08T06:04:03.553892+0000","last_became_active":"2024-03-08T06:04:02.545953+0000","last_became_peered":"2024-03-08T06:04:02.545953+0000","last_unstale":"2024-03-08T06:04:03.553892+0000","last_undegraded":"2024-03-08T06:04:03.553892+0000","last_fullsized":"2024-03-08T06:04:03.553892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.550165+0000","last_change":"2024-03-08T06:04:02.544781+0000","last_active":"2024-03-08T06:04:02.550165+0000","last_peered":"2024-03-08T06:04:02.550165+0000","last_clean":"2024-03-08T06:04:02.550165+0000","last_became_active":"2024-03-08T06:04:02.543995+0000","last_became_peered":"2024-03-08T06:04:02.543995+0000","last_unstale":"2024-03-08T06:04:02.550165+0000","last_undegraded":"2024-03-08T06:04:02.550165+0000","last_fullsized":"2024-03-08T06:04:02.550165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:59.913059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515851+0000","last_change":"2024-03-08T06:04:02.551644+0000","last_active":"2024-03-08T06:04:03.515851+0000","last_peered":"2024-03-08T06:04:03.515851+0000","last_clean":"2024-03-08T06:04:03.515851+0000","last_became_active":"2024-03-08T06:04:02.550930+0000","last_became_peered":"2024-03-08T06:04:02.550930+0000","last_unstale":"2024-03-08T06:04:03.515851+0000","last_undegraded":"2024-03-08T06:04:03.515851+0000","last_fullsized":"2024-03-08T06:04:03.515851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:23:55.655734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617514+0000","last_change":"2024-03-08T06:04:02.545560+0000","last_active":"2024-03-08T06:04:03.617514+0000","last_peered":"2024-03-08T06:04:03.617514+0000","last_clean":"2024-03-08T06:04:03.617514+0000","last_became_active":"2024-03-08T06:04:02.543855+0000","last_became_peered":"2024-03-08T06:04:02.543855+0000","last_unstale":"2024-03-08T06:04:03.617514+0000","last_undegraded":"2024-03-08T06:04:03.617514+0000","last_fullsized":"2024-03-08T06:04:03.617514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515916+0000","last_change":"2024-03-08T06:04:02.550172+0000","last_active":"2024-03-08T06:04:03.515916+0000","last_peered":"2024-03-08T06:04:03.515916+0000","last_clean":"2024-03-08T06:04:03.515916+0000","last_became_active":"2024-03-08T06:04:02.549807+0000","last_became_peered":"2024-03-08T06:04:02.549807+0000","last_unstale":"2024-03-08T06:04:03.515916+0000","last_undegraded":"2024-03-08T06:04:03.515916+0000","last_fullsized":"2024-03-08T06:04:03.515916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617257+0000","last_change":"2024-03-08T06:04:02.547691+0000","last_active":"2024-03-08T06:04:03.617257+0000","last_peered":"2024-03-08T06:04:03.617257+0000","last_clean":"2024-03-08T06:04:03.617257+0000","last_became_active":"2024-03-08T06:04:02.547325+0000","last_became_peered":"2024-03-08T06:04:02.547325+0000","last_unstale":"2024-03-08T06:04:03.617257+0000","last_undegraded":"2024-03-08T06:04:03.617257+0000","last_fullsized":"2024-03-08T06:04:03.617257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511377+0000","last_change":"2024-03-08T06:04:02.553110+0000","last_active":"2024-03-08T06:04:03.511377+0000","last_peered":"2024-03-08T06:04:03.511377+0000","last_clean":"2024-03-08T06:04:03.511377+0000","last_became_active":"2024-03-08T06:04:02.552635+0000","last_became_peered":"2024-03-08T06:04:02.552635+0000","last_unstale":"2024-03-08T06:04:03.511377+0000","last_undegraded":"2024-03-08T06:04:03.511377+0000","last_fullsized":"2024-03-08T06:04:03.511377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618751+0000","last_change":"2024-03-08T06:04:02.570801+0000","last_active":"2024-03-08T06:04:03.618751+0000","last_peered":"2024-03-08T06:04:03.618751+0000","last_clean":"2024-03-08T06:04:03.618751+0000","last_became_active":"2024-03-08T06:04:02.570558+0000","last_became_peered":"2024-03-08T06:04:02.570558+0000","last_unstale":"2024-03-08T06:04:03.618751+0000","last_undegraded":"2024-03-08T06:04:03.618751+0000","last_fullsized":"2024-03-08T06:04:03.618751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:07:57.853863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559891+0000","last_change":"2024-03-08T06:04:02.562132+0000","last_active":"2024-03-08T06:04:03.559891+0000","last_peered":"2024-03-08T06:04:03.559891+0000","last_clean":"2024-03-08T06:04:03.559891+0000","last_became_active":"2024-03-08T06:04:02.561851+0000","last_became_peered":"2024-03-08T06:04:02.561851+0000","last_unstale":"2024-03-08T06:04:03.559891+0000","last_undegraded":"2024-03-08T06:04:03.559891+0000","last_fullsized":"2024-03-08T06:04:03.559891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559579+0000","last_change":"2024-03-08T06:04:02.546846+0000","last_active":"2024-03-08T06:04:03.559579+0000","last_peered":"2024-03-08T06:04:03.559579+0000","last_clean":"2024-03-08T06:04:03.559579+0000","last_became_active":"2024-03-08T06:04:02.546522+0000","last_became_peered":"2024-03-08T06:04:02.546522+0000","last_unstale":"2024-03-08T06:04:03.559579+0000","last_undegraded":"2024-03-08T06:04:03.559579+0000","last_fullsized":"2024-03-08T06:04:03.559579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118422+0000","last_change":"2024-03-08T06:04:02.555942+0000","last_active":"2024-03-08T06:04:04.118422+0000","last_peered":"2024-03-08T06:04:04.118422+0000","last_clean":"2024-03-08T06:04:04.118422+0000","last_became_active":"2024-03-08T06:04:02.555651+0000","last_became_peered":"2024-03-08T06:04:02.555651+0000","last_unstale":"2024-03-08T06:04:04.118422+0000","last_undegraded":"2024-03-08T06:04:04.118422+0000","last_fullsized":"2024-03-08T06:04:04.118422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:57.032278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511828+0000","last_change":"2024-03-08T06:04:02.551969+0000","last_active":"2024-03-08T06:04:03.511828+0000","last_peered":"2024-03-08T06:04:03.511828+0000","last_clean":"2024-03-08T06:04:03.511828+0000","last_became_active":"2024-03-08T06:04:02.551664+0000","last_became_peered":"2024-03-08T06:04:02.551664+0000","last_unstale":"2024-03-08T06:04:03.511828+0000","last_undegraded":"2024-03-08T06:04:03.511828+0000","last_fullsized":"2024-03-08T06:04:03.511828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.547045+0000","last_change":"2024-03-08T06:04:02.542608+0000","last_active":"2024-03-08T06:04:02.547045+0000","last_peered":"2024-03-08T06:04:02.547045+0000","last_clean":"2024-03-08T06:04:02.547045+0000","last_became_active":"2024-03-08T06:04:02.542318+0000","last_became_peered":"2024-03-08T06:04:02.542318+0000","last_unstale":"2024-03-08T06:04:02.547045+0000","last_undegraded":"2024-03-08T06:04:02.547045+0000","last_fullsized":"2024-03-08T06:04:02.547045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:33.768388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553245+0000","last_change":"2024-03-08T06:04:02.548664+0000","last_active":"2024-03-08T06:04:03.553245+0000","last_peered":"2024-03-08T06:04:03.553245+0000","last_clean":"2024-03-08T06:04:03.553245+0000","last_became_active":"2024-03-08T06:04:02.547601+0000","last_became_peered":"2024-03-08T06:04:02.547601+0000","last_unstale":"2024-03-08T06:04:03.553245+0000","last_undegraded":"2024-03-08T06:04:03.553245+0000","last_fullsized":"2024-03-08T06:04:03.553245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515232+0000","last_change":"2024-03-08T06:04:02.551648+0000","last_active":"2024-03-08T06:04:03.515232+0000","last_peered":"2024-03-08T06:04:03.515232+0000","last_clean":"2024-03-08T06:04:03.515232+0000","last_became_active":"2024-03-08T06:04:02.551290+0000","last_became_peered":"2024-03-08T06:04:02.551290+0000","last_unstale":"2024-03-08T06:04:03.515232+0000","last_undegraded":"2024-03-08T06:04:03.515232+0000","last_fullsized":"2024-03-08T06:04:03.515232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.518022+0000","last_change":"2024-03-08T06:04:02.556704+0000","last_active":"2024-03-08T06:04:03.518022+0000","last_peered":"2024-03-08T06:04:03.518022+0000","last_clean":"2024-03-08T06:04:03.518022+0000","last_became_active":"2024-03-08T06:04:02.556429+0000","last_became_peered":"2024-03-08T06:04:02.556429+0000","last_unstale":"2024-03-08T06:04:03.518022+0000","last_undegraded":"2024-03-08T06:04:03.518022+0000","last_fullsized":"2024-03-08T06:04:03.518022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:13:36.393811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.543918+0000","last_change":"2024-03-08T06:04:02.539066+0000","last_active":"2024-03-08T06:04:02.543918+0000","last_peered":"2024-03-08T06:04:02.543918+0000","last_clean":"2024-03-08T06:04:02.543918+0000","last_became_active":"2024-03-08T06:04:02.538745+0000","last_became_peered":"2024-03-08T06:04:02.538745+0000","last_unstale":"2024-03-08T06:04:02.543918+0000","last_undegraded":"2024-03-08T06:04:02.543918+0000","last_fullsized":"2024-03-08T06:04:02.543918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:52.535460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618590+0000","last_change":"2024-03-08T06:04:02.551987+0000","last_active":"2024-03-08T06:04:03.618590+0000","last_peered":"2024-03-08T06:04:03.618590+0000","last_clean":"2024-03-08T06:04:03.618590+0000","last_became_active":"2024-03-08T06:04:02.551425+0000","last_became_peered":"2024-03-08T06:04:02.551425+0000","last_unstale":"2024-03-08T06:04:03.618590+0000","last_undegraded":"2024-03-08T06:04:03.618590+0000","last_fullsized":"2024-03-08T06:04:03.618590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:41:43.397781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559135+0000","last_change":"2024-03-08T06:04:02.559334+0000","last_active":"2024-03-08T06:04:03.559135+0000","last_peered":"2024-03-08T06:04:03.559135+0000","last_clean":"2024-03-08T06:04:03.559135+0000","last_became_active":"2024-03-08T06:04:02.558979+0000","last_became_peered":"2024-03-08T06:04:02.558979+0000","last_unstale":"2024-03-08T06:04:03.559135+0000","last_undegraded":"2024-03-08T06:04:03.559135+0000","last_fullsized":"2024-03-08T06:04:03.559135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.550247+0000","last_change":"2024-03-08T06:04:02.545858+0000","last_active":"2024-03-08T06:04:02.550247+0000","last_peered":"2024-03-08T06:04:02.550247+0000","last_clean":"2024-03-08T06:04:02.550247+0000","last_became_active":"2024-03-08T06:04:02.545518+0000","last_became_peered":"2024-03-08T06:04:02.545518+0000","last_unstale":"2024-03-08T06:04:02.550247+0000","last_undegraded":"2024-03-08T06:04:02.550247+0000","last_fullsized":"2024-03-08T06:04:02.550247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:19.592493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516005+0000","last_change":"2024-03-08T06:04:02.553462+0000","last_active":"2024-03-08T06:04:03.516005+0000","last_peered":"2024-03-08T06:04:03.516005+0000","last_clean":"2024-03-08T06:04:03.516005+0000","last_became_active":"2024-03-08T06:04:02.552688+0000","last_became_peered":"2024-03-08T06:04:02.552688+0000","last_unstale":"2024-03-08T06:04:03.516005+0000","last_undegraded":"2024-03-08T06:04:03.516005+0000","last_fullsized":"2024-03-08T06:04:03.516005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511690+0000","last_change":"2024-03-08T06:04:02.560361+0000","last_active":"2024-03-08T06:04:03.511690+0000","last_peered":"2024-03-08T06:04:03.511690+0000","last_clean":"2024-03-08T06:04:03.511690+0000","last_became_active":"2024-03-08T06:04:02.560083+0000","last_became_peered":"2024-03-08T06:04:02.560083+0000","last_unstale":"2024-03-08T06:04:03.511690+0000","last_undegraded":"2024-03-08T06:04:03.511690+0000","last_fullsized":"2024-03-08T06:04:03.511690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515403+0000","last_change":"2024-03-08T06:04:02.549163+0000","last_active":"2024-03-08T06:04:03.515403+0000","last_peered":"2024-03-08T06:04:03.515403+0000","last_clean":"2024-03-08T06:04:03.515403+0000","last_became_active":"2024-03-08T06:04:02.548715+0000","last_became_peered":"2024-03-08T06:04:02.548715+0000","last_unstale":"2024-03-08T06:04:03.515403+0000","last_undegraded":"2024-03-08T06:04:03.515403+0000","last_fullsized":"2024-03-08T06:04:03.515403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512314+0000","last_change":"2024-03-08T06:04:02.565662+0000","last_active":"2024-03-08T06:04:03.512314+0000","last_peered":"2024-03-08T06:04:03.512314+0000","last_clean":"2024-03-08T06:04:03.512314+0000","last_became_active":"2024-03-08T06:04:02.565344+0000","last_became_peered":"2024-03-08T06:04:02.565344+0000","last_unstale":"2024-03-08T06:04:03.512314+0000","last_undegraded":"2024-03-08T06:04:03.512314+0000","last_fullsized":"2024-03-08T06:04:03.512314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:23:55.655734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.560225+0000","last_change":"2024-03-08T06:04:02.545865+0000","last_active":"2024-03-08T06:04:03.560225+0000","last_peered":"2024-03-08T06:04:03.560225+0000","last_clean":"2024-03-08T06:04:03.560225+0000","last_became_active":"2024-03-08T06:04:02.543828+0000","last_became_peered":"2024-03-08T06:04:02.543828+0000","last_unstale":"2024-03-08T06:04:03.560225+0000","last_undegraded":"2024-03-08T06:04:03.560225+0000","last_fullsized":"2024-03-08T06:04:03.560225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553769+0000","last_change":"2024-03-08T06:04:02.544859+0000","last_active":"2024-03-08T06:04:03.553769+0000","last_peered":"2024-03-08T06:04:03.553769+0000","last_clean":"2024-03-08T06:04:03.553769+0000","last_became_active":"2024-03-08T06:04:02.544502+0000","last_became_peered":"2024-03-08T06:04:02.544502+0000","last_unstale":"2024-03-08T06:04:03.553769+0000","last_undegraded":"2024-03-08T06:04:03.553769+0000","last_fullsized":"2024-03-08T06:04:03.553769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512315+0000","last_change":"2024-03-08T06:04:02.563733+0000","last_active":"2024-03-08T06:04:03.512315+0000","last_peered":"2024-03-08T06:04:03.512315+0000","last_clean":"2024-03-08T06:04:03.512315+0000","last_became_active":"2024-03-08T06:04:02.563420+0000","last_became_peered":"2024-03-08T06:04:02.563420+0000","last_unstale":"2024-03-08T06:04:03.512315+0000","last_undegraded":"2024-03-08T06:04:03.512315+0000","last_fullsized":"2024-03-08T06:04:03.512315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.551336+0000","last_change":"2024-03-08T06:04:02.548150+0000","last_active":"2024-03-08T06:04:02.551336+0000","last_peered":"2024-03-08T06:04:02.551336+0000","last_clean":"2024-03-08T06:04:02.551336+0000","last_became_active":"2024-03-08T06:04:02.547846+0000","last_became_peered":"2024-03-08T06:04:02.547846+0000","last_unstale":"2024-03-08T06:04:02.551336+0000","last_undegraded":"2024-03-08T06:04:02.551336+0000","last_fullsized":"2024-03-08T06:04:02.551336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:52.535460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516515+0000","last_change":"2024-03-08T06:04:02.547204+0000","last_active":"2024-03-08T06:04:03.516515+0000","last_peered":"2024-03-08T06:04:03.516515+0000","last_clean":"2024-03-08T06:04:03.516515+0000","last_became_active":"2024-03-08T06:04:02.546850+0000","last_became_peered":"2024-03-08T06:04:02.546850+0000","last_unstale":"2024-03-08T06:04:03.516515+0000","last_undegraded":"2024-03-08T06:04:03.516515+0000","last_fullsized":"2024-03-08T06:04:03.516515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:32:53.947147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.559191+0000","last_change":"2024-03-08T06:04:02.553107+0000","last_active":"2024-03-08T06:04:02.559191+0000","last_peered":"2024-03-08T06:04:02.559191+0000","last_clean":"2024-03-08T06:04:02.559191+0000","last_became_active":"2024-03-08T06:04:02.552862+0000","last_became_peered":"2024-03-08T06:04:02.552862+0000","last_unstale":"2024-03-08T06:04:02.559191+0000","last_undegraded":"2024-03-08T06:04:02.559191+0000","last_fullsized":"2024-03-08T06:04:02.559191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:09.964322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559013+0000","last_change":"2024-03-08T06:04:02.544639+0000","last_active":"2024-03-08T06:04:03.559013+0000","last_peered":"2024-03-08T06:04:03.559013+0000","last_clean":"2024-03-08T06:04:03.559013+0000","last_became_active":"2024-03-08T06:04:02.544193+0000","last_became_peered":"2024-03-08T06:04:02.544193+0000","last_unstale":"2024-03-08T06:04:03.559013+0000","last_undegraded":"2024-03-08T06:04:03.559013+0000","last_fullsized":"2024-03-08T06:04:03.559013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.117892+0000","last_change":"2024-03-08T06:04:02.558328+0000","last_active":"2024-03-08T06:04:04.117892+0000","last_peered":"2024-03-08T06:04:04.117892+0000","last_clean":"2024-03-08T06:04:04.117892+0000","last_became_active":"2024-03-08T06:04:02.555015+0000","last_became_peered":"2024-03-08T06:04:02.555015+0000","last_unstale":"2024-03-08T06:04:04.117892+0000","last_undegraded":"2024-03-08T06:04:04.117892+0000","last_fullsized":"2024-03-08T06:04:04.117892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:52.535460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512099+0000","last_change":"2024-03-08T06:04:01.555471+0000","last_active":"2024-03-08T06:04:03.512099+0000","last_peered":"2024-03-08T06:04:03.512099+0000","last_clean":"2024-03-08T06:04:03.512099+0000","last_became_active":"2024-03-08T06:04:01.555171+0000","last_became_peered":"2024-03-08T06:04:01.555171+0000","last_unstale":"2024-03-08T06:04:03.512099+0000","last_undegraded":"2024-03-08T06:04:03.512099+0000","last_fullsized":"2024-03-08T06:04:03.512099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.566828+0000","last_change":"2024-03-08T06:04:02.563404+0000","last_active":"2024-03-08T06:04:02.566828+0000","last_peered":"2024-03-08T06:04:02.566828+0000","last_clean":"2024-03-08T06:04:02.566828+0000","last_became_active":"2024-03-08T06:04:02.558959+0000","last_became_peered":"2024-03-08T06:04:02.558959+0000","last_unstale":"2024-03-08T06:04:02.566828+0000","last_undegraded":"2024-03-08T06:04:02.566828+0000","last_fullsized":"2024-03-08T06:04:02.566828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:06:00.048492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559117+0000","last_change":"2024-03-08T06:04:01.549647+0000","last_active":"2024-03-08T06:04:03.559117+0000","last_peered":"2024-03-08T06:04:03.559117+0000","last_clean":"2024-03-08T06:04:03.559117+0000","last_became_active":"2024-03-08T06:04:01.549345+0000","last_became_peered":"2024-03-08T06:04:01.549345+0000","last_unstale":"2024-03-08T06:04:03.559117+0000","last_undegraded":"2024-03-08T06:04:03.559117+0000","last_fullsized":"2024-03-08T06:04:03.559117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511841+0000","last_change":"2024-03-08T06:04:01.551862+0000","last_active":"2024-03-08T06:04:03.511841+0000","last_peered":"2024-03-08T06:04:03.511841+0000","last_clean":"2024-03-08T06:04:03.511841+0000","last_became_active":"2024-03-08T06:04:01.551606+0000","last_became_peered":"2024-03-08T06:04:01.551606+0000","last_unstale":"2024-03-08T06:04:03.511841+0000","last_undegraded":"2024-03-08T06:04:03.511841+0000","last_fullsized":"2024-03-08T06:04:03.511841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:52:36.740849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512342+0000","last_change":"2024-03-08T06:04:02.548972+0000","last_active":"2024-03-08T06:04:03.512342+0000","last_peered":"2024-03-08T06:04:03.512342+0000","last_clean":"2024-03-08T06:04:03.512342+0000","last_became_active":"2024-03-08T06:04:02.548136+0000","last_became_peered":"2024-03-08T06:04:02.548136+0000","last_unstale":"2024-03-08T06:04:03.512342+0000","last_undegraded":"2024-03-08T06:04:03.512342+0000","last_fullsized":"2024-03-08T06:04:03.512342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516175+0000","last_change":"2024-03-08T06:04:01.556755+0000","last_active":"2024-03-08T06:04:03.516175+0000","last_peered":"2024-03-08T06:04:03.516175+0000","last_clean":"2024-03-08T06:04:03.516175+0000","last_became_active":"2024-03-08T06:04:01.555842+0000","last_became_peered":"2024-03-08T06:04:01.555842+0000","last_unstale":"2024-03-08T06:04:03.516175+0000","last_undegraded":"2024-03-08T06:04:03.516175+0000","last_fullsized":"2024-03-08T06:04:03.516175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553244+0000","last_change":"2024-03-08T06:04:02.549137+0000","last_active":"2024-03-08T06:04:03.553244+0000","last_peered":"2024-03-08T06:04:03.553244+0000","last_clean":"2024-03-08T06:04:03.553244+0000","last_became_active":"2024-03-08T06:04:02.547990+0000","last_became_peered":"2024-03-08T06:04:02.547990+0000","last_unstale":"2024-03-08T06:04:03.553244+0000","last_undegraded":"2024-03-08T06:04:03.553244+0000","last_fullsized":"2024-03-08T06:04:03.553244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559731+0000","last_change":"2024-03-08T06:04:01.555948+0000","last_active":"2024-03-08T06:04:03.559731+0000","last_peered":"2024-03-08T06:04:03.559731+0000","last_clean":"2024-03-08T06:04:03.559731+0000","last_became_active":"2024-03-08T06:04:01.555666+0000","last_became_peered":"2024-03-08T06:04:01.555666+0000","last_unstale":"2024-03-08T06:04:03.559731+0000","last_undegraded":"2024-03-08T06:04:03.559731+0000","last_fullsized":"2024-03-08T06:04:03.559731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517528+0000","last_change":"2024-03-08T06:04:02.558089+0000","last_active":"2024-03-08T06:04:03.517528+0000","last_peered":"2024-03-08T06:04:03.517528+0000","last_clean":"2024-03-08T06:04:03.517528+0000","last_became_active":"2024-03-08T06:04:02.557697+0000","last_became_peered":"2024-03-08T06:04:02.557697+0000","last_unstale":"2024-03-08T06:04:03.517528+0000","last_undegraded":"2024-03-08T06:04:03.517528+0000","last_fullsized":"2024-03-08T06:04:03.517528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:19:21.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517312+0000","last_change":"2024-03-08T06:04:01.559428+0000","last_active":"2024-03-08T06:04:03.517312+0000","last_peered":"2024-03-08T06:04:03.517312+0000","last_clean":"2024-03-08T06:04:03.517312+0000","last_became_active":"2024-03-08T06:04:01.559156+0000","last_became_peered":"2024-03-08T06:04:01.559156+0000","last_unstale":"2024-03-08T06:04:03.517312+0000","last_undegraded":"2024-03-08T06:04:03.517312+0000","last_fullsized":"2024-03-08T06:04:03.517312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.564662+0000","last_change":"2024-03-08T06:04:02.551412+0000","last_active":"2024-03-08T06:04:02.564662+0000","last_peered":"2024-03-08T06:04:02.564662+0000","last_clean":"2024-03-08T06:04:02.564662+0000","last_became_active":"2024-03-08T06:04:02.550080+0000","last_became_peered":"2024-03-08T06:04:02.550080+0000","last_unstale":"2024-03-08T06:04:02.564662+0000","last_undegraded":"2024-03-08T06:04:02.564662+0000","last_fullsized":"2024-03-08T06:04:02.564662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:51.718653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553697+0000","last_change":"2024-03-08T06:04:01.554654+0000","last_active":"2024-03-08T06:04:03.553697+0000","last_peered":"2024-03-08T06:04:03.553697+0000","last_clean":"2024-03-08T06:04:03.553697+0000","last_became_active":"2024-03-08T06:04:01.554238+0000","last_became_peered":"2024-03-08T06:04:01.554238+0000","last_unstale":"2024-03-08T06:04:03.553697+0000","last_undegraded":"2024-03-08T06:04:03.553697+0000","last_fullsized":"2024-03-08T06:04:03.553697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118544+0000","last_change":"2024-03-08T06:04:02.556415+0000","last_active":"2024-03-08T06:04:04.118544+0000","last_peered":"2024-03-08T06:04:04.118544+0000","last_clean":"2024-03-08T06:04:04.118544+0000","last_became_active":"2024-03-08T06:04:02.556055+0000","last_became_peered":"2024-03-08T06:04:02.556055+0000","last_unstale":"2024-03-08T06:04:04.118544+0000","last_undegraded":"2024-03-08T06:04:04.118544+0000","last_fullsized":"2024-03-08T06:04:04.118544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:06:00.048492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559371+0000","last_change":"2024-03-08T06:04:01.549872+0000","last_active":"2024-03-08T06:04:03.559371+0000","last_peered":"2024-03-08T06:04:03.559371+0000","last_clean":"2024-03-08T06:04:03.559371+0000","last_became_active":"2024-03-08T06:04:01.549313+0000","last_became_peered":"2024-03-08T06:04:01.549313+0000","last_unstale":"2024-03-08T06:04:03.559371+0000","last_undegraded":"2024-03-08T06:04:03.559371+0000","last_fullsized":"2024-03-08T06:04:03.559371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618023+0000","last_change":"2024-03-08T06:04:02.549539+0000","last_active":"2024-03-08T06:04:03.618023+0000","last_peered":"2024-03-08T06:04:03.618023+0000","last_clean":"2024-03-08T06:04:03.618023+0000","last_became_active":"2024-03-08T06:04:02.549245+0000","last_became_peered":"2024-03-08T06:04:02.549245+0000","last_unstale":"2024-03-08T06:04:03.618023+0000","last_undegraded":"2024-03-08T06:04:03.618023+0000","last_fullsized":"2024-03-08T06:04:03.618023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:33:11.525956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.574228+0000","last_change":"2024-03-08T06:04:01.542685+0000","last_active":"2024-03-08T06:04:02.574228+0000","last_peered":"2024-03-08T06:04:02.574228+0000","last_clean":"2024-03-08T06:04:02.574228+0000","last_became_active":"2024-03-08T06:04:01.542236+0000","last_became_peered":"2024-03-08T06:04:01.542236+0000","last_unstale":"2024-03-08T06:04:02.574228+0000","last_undegraded":"2024-03-08T06:04:02.574228+0000","last_fullsized":"2024-03-08T06:04:02.574228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:35:24.125408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.564748+0000","last_change":"2024-03-08T06:04:02.553210+0000","last_active":"2024-03-08T06:04:02.564748+0000","last_peered":"2024-03-08T06:04:02.564748+0000","last_clean":"2024-03-08T06:04:02.564748+0000","last_became_active":"2024-03-08T06:04:02.552889+0000","last_became_peered":"2024-03-08T06:04:02.552889+0000","last_unstale":"2024-03-08T06:04:02.564748+0000","last_undegraded":"2024-03-08T06:04:02.564748+0000","last_fullsized":"2024-03-08T06:04:02.564748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:57.032278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516402+0000","last_change":"2024-03-08T06:04:01.554506+0000","last_active":"2024-03-08T06:04:03.516402+0000","last_peered":"2024-03-08T06:04:03.516402+0000","last_clean":"2024-03-08T06:04:03.516402+0000","last_became_active":"2024-03-08T06:04:01.554224+0000","last_became_peered":"2024-03-08T06:04:01.554224+0000","last_unstale":"2024-03-08T06:04:03.516402+0000","last_undegraded":"2024-03-08T06:04:03.516402+0000","last_fullsized":"2024-03-08T06:04:03.516402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617833+0000","last_change":"2024-03-08T06:04:02.560754+0000","last_active":"2024-03-08T06:04:03.617833+0000","last_peered":"2024-03-08T06:04:03.617833+0000","last_clean":"2024-03-08T06:04:03.617833+0000","last_became_active":"2024-03-08T06:04:02.560482+0000","last_became_peered":"2024-03-08T06:04:02.560482+0000","last_unstale":"2024-03-08T06:04:03.617833+0000","last_undegraded":"2024-03-08T06:04:03.617833+0000","last_fullsized":"2024-03-08T06:04:03.617833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:32:53.947147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559130+0000","last_change":"2024-03-08T06:04:01.545961+0000","last_active":"2024-03-08T06:04:03.559130+0000","last_peered":"2024-03-08T06:04:03.559130+0000","last_clean":"2024-03-08T06:04:03.559130+0000","last_became_active":"2024-03-08T06:04:01.545579+0000","last_became_peered":"2024-03-08T06:04:01.545579+0000","last_unstale":"2024-03-08T06:04:03.559130+0000","last_undegraded":"2024-03-08T06:04:03.559130+0000","last_fullsized":"2024-03-08T06:04:03.559130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.558973+0000","last_change":"2024-03-08T06:04:02.544735+0000","last_active":"2024-03-08T06:04:03.558973+0000","last_peered":"2024-03-08T06:04:03.558973+0000","last_clean":"2024-03-08T06:04:03.558973+0000","last_became_active":"2024-03-08T06:04:02.544438+0000","last_became_peered":"2024-03-08T06:04:02.544438+0000","last_unstale":"2024-03-08T06:04:03.558973+0000","last_undegraded":"2024-03-08T06:04:03.558973+0000","last_fullsized":"2024-03-08T06:04:03.558973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.568401+0000","last_change":"2024-03-08T06:04:01.541060+0000","last_active":"2024-03-08T06:04:02.568401+0000","last_peered":"2024-03-08T06:04:02.568401+0000","last_clean":"2024-03-08T06:04:02.568401+0000","last_became_active":"2024-03-08T06:04:01.540657+0000","last_became_peered":"2024-03-08T06:04:01.540657+0000","last_unstale":"2024-03-08T06:04:02.568401+0000","last_undegraded":"2024-03-08T06:04:02.568401+0000","last_fullsized":"2024-03-08T06:04:02.568401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617381+0000","last_change":"2024-03-08T06:04:02.565226+0000","last_active":"2024-03-08T06:04:03.617381+0000","last_peered":"2024-03-08T06:04:03.617381+0000","last_clean":"2024-03-08T06:04:03.617381+0000","last_became_active":"2024-03-08T06:04:02.564692+0000","last_became_peered":"2024-03-08T06:04:02.564692+0000","last_unstale":"2024-03-08T06:04:03.617381+0000","last_undegraded":"2024-03-08T06:04:03.617381+0000","last_fullsized":"2024-03-08T06:04:03.617381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118978+0000","last_change":"2024-03-08T06:04:01.540655+0000","last_active":"2024-03-08T06:04:04.118978+0000","last_peered":"2024-03-08T06:04:04.118978+0000","last_clean":"2024-03-08T06:04:04.118978+0000","last_became_active":"2024-03-08T06:04:01.540205+0000","last_became_peered":"2024-03-08T06:04:01.540205+0000","last_unstale":"2024-03-08T06:04:04.118978+0000","last_undegraded":"2024-03-08T06:04:04.118978+0000","last_fullsized":"2024-03-08T06:04:04.118978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:12:32.122620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618419+0000","last_change":"2024-03-08T06:04:02.559001+0000","last_active":"2024-03-08T06:04:03.618419+0000","last_peered":"2024-03-08T06:04:03.618419+0000","last_clean":"2024-03-08T06:04:03.618419+0000","last_became_active":"2024-03-08T06:04:02.558681+0000","last_became_peered":"2024-03-08T06:04:02.558681+0000","last_unstale":"2024-03-08T06:04:03.618419+0000","last_undegraded":"2024-03-08T06:04:03.618419+0000","last_fullsized":"2024-03-08T06:04:03.618419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:19:21.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.570018+0000","last_change":"2024-03-08T06:04:01.542669+0000","last_active":"2024-03-08T06:04:02.570018+0000","last_peered":"2024-03-08T06:04:02.570018+0000","last_clean":"2024-03-08T06:04:02.570018+0000","last_became_active":"2024-03-08T06:04:01.540916+0000","last_became_peered":"2024-03-08T06:04:01.540916+0000","last_unstale":"2024-03-08T06:04:02.570018+0000","last_undegraded":"2024-03-08T06:04:02.570018+0000","last_fullsized":"2024-03-08T06:04:02.570018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618960+0000","last_change":"2024-03-08T06:04:02.569733+0000","last_active":"2024-03-08T06:04:03.618960+0000","last_peered":"2024-03-08T06:04:03.618960+0000","last_clean":"2024-03-08T06:04:03.618960+0000","last_became_active":"2024-03-08T06:04:02.567299+0000","last_became_peered":"2024-03-08T06:04:02.567299+0000","last_unstale":"2024-03-08T06:04:03.618960+0000","last_undegraded":"2024-03-08T06:04:03.618960+0000","last_fullsized":"2024-03-08T06:04:03.618960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:16:15.668237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553839+0000","last_change":"2024-03-08T06:04:01.544281+0000","last_active":"2024-03-08T06:04:03.553839+0000","last_peered":"2024-03-08T06:04:03.553839+0000","last_clean":"2024-03-08T06:04:03.553839+0000","last_became_active":"2024-03-08T06:04:01.543982+0000","last_became_peered":"2024-03-08T06:04:01.543982+0000","last_unstale":"2024-03-08T06:04:03.553839+0000","last_undegraded":"2024-03-08T06:04:03.553839+0000","last_fullsized":"2024-03-08T06:04:03.553839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:50.471516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516800+0000","last_change":"2024-03-08T06:04:02.551888+0000","last_active":"2024-03-08T06:04:03.516800+0000","last_peered":"2024-03-08T06:04:03.516800+0000","last_clean":"2024-03-08T06:04:03.516800+0000","last_became_active":"2024-03-08T06:04:02.551321+0000","last_became_peered":"2024-03-08T06:04:02.551321+0000","last_unstale":"2024-03-08T06:04:03.516800+0000","last_undegraded":"2024-03-08T06:04:03.516800+0000","last_fullsized":"2024-03-08T06:04:03.516800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:33:11.525956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.512617+0000","last_change":"2024-03-08T06:04:01.554656+0000","last_active":"2024-03-08T06:04:02.512617+0000","last_peered":"2024-03-08T06:04:02.512617+0000","last_clean":"2024-03-08T06:04:02.512617+0000","last_became_active":"2024-03-08T06:04:01.554326+0000","last_became_peered":"2024-03-08T06:04:01.554326+0000","last_unstale":"2024-03-08T06:04:02.512617+0000","last_undegraded":"2024-03-08T06:04:02.512617+0000","last_fullsized":"2024-03-08T06:04:02.512617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618096+0000","last_change":"2024-03-08T06:04:02.568164+0000","last_active":"2024-03-08T06:04:03.618096+0000","last_peered":"2024-03-08T06:04:03.618096+0000","last_clean":"2024-03-08T06:04:03.618096+0000","last_became_active":"2024-03-08T06:04:02.567123+0000","last_became_peered":"2024-03-08T06:04:02.567123+0000","last_unstale":"2024-03-08T06:04:03.618096+0000","last_undegraded":"2024-03-08T06:04:03.618096+0000","last_fullsized":"2024-03-08T06:04:03.618096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:42:12.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617235+0000","last_change":"2024-03-08T06:03:57.473597+0000","last_active":"2024-03-08T06:04:03.617235+0000","last_peered":"2024-03-08T06:04:03.617235+0000","last_clean":"2024-03-08T06:04:03.617235+0000","last_became_active":"2024-03-08T06:03:57.468593+0000","last_became_peered":"2024-03-08T06:03:57.468593+0000","last_unstale":"2024-03-08T06:04:03.617235+0000","last_undegraded":"2024-03-08T06:04:03.617235+0000","last_fullsized":"2024-03-08T06:04:03.617235+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_clean_scrub_stamp":"2024-03-08T06:03:56.214441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:45:28.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515695+0000","last_change":"2024-03-08T06:04:01.554640+0000","last_active":"2024-03-08T06:04:03.515695+0000","last_peered":"2024-03-08T06:04:03.515695+0000","last_clean":"2024-03-08T06:04:03.515695+0000","last_became_active":"2024-03-08T06:04:01.554274+0000","last_became_peered":"2024-03-08T06:04:01.554274+0000","last_unstale":"2024-03-08T06:04:03.515695+0000","last_undegraded":"2024-03-08T06:04:03.515695+0000","last_fullsized":"2024-03-08T06:04:03.515695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511891+0000","last_change":"2024-03-08T06:04:02.556830+0000","last_active":"2024-03-08T06:04:03.511891+0000","last_peered":"2024-03-08T06:04:03.511891+0000","last_clean":"2024-03-08T06:04:03.511891+0000","last_became_active":"2024-03-08T06:04:02.556071+0000","last_became_peered":"2024-03-08T06:04:02.556071+0000","last_unstale":"2024-03-08T06:04:03.511891+0000","last_undegraded":"2024-03-08T06:04:03.511891+0000","last_fullsized":"2024-03-08T06:04:03.511891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617015+0000","last_change":"2024-03-08T06:04:01.556875+0000","last_active":"2024-03-08T06:04:03.617015+0000","last_peered":"2024-03-08T06:04:03.617015+0000","last_clean":"2024-03-08T06:04:03.617015+0000","last_became_active":"2024-03-08T06:04:01.555346+0000","last_became_peered":"2024-03-08T06:04:01.555346+0000","last_unstale":"2024-03-08T06:04:03.617015+0000","last_undegraded":"2024-03-08T06:04:03.617015+0000","last_fullsized":"2024-03-08T06:04:03.617015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.566425+0000","last_change":"2024-03-08T06:04:02.563977+0000","last_active":"2024-03-08T06:04:02.566425+0000","last_peered":"2024-03-08T06:04:02.566425+0000","last_clean":"2024-03-08T06:04:02.566425+0000","last_became_active":"2024-03-08T06:04:02.563653+0000","last_became_peered":"2024-03-08T06:04:02.563653+0000","last_unstale":"2024-03-08T06:04:02.566425+0000","last_undegraded":"2024-03-08T06:04:02.566425+0000","last_fullsized":"2024-03-08T06:04:02.566425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:23:12.541838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.514447+0000","last_change":"2024-03-08T06:04:01.542592+0000","last_active":"2024-03-08T06:04:02.514447+0000","last_peered":"2024-03-08T06:04:02.514447+0000","last_clean":"2024-03-08T06:04:02.514447+0000","last_became_active":"2024-03-08T06:04:01.541567+0000","last_became_peered":"2024-03-08T06:04:01.541567+0000","last_unstale":"2024-03-08T06:04:02.514447+0000","last_undegraded":"2024-03-08T06:04:02.514447+0000","last_fullsized":"2024-03-08T06:04:02.514447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553855+0000","last_change":"2024-03-08T06:04:02.525242+0000","last_active":"2024-03-08T06:04:03.553855+0000","last_peered":"2024-03-08T06:04:03.553855+0000","last_clean":"2024-03-08T06:04:03.553855+0000","last_became_active":"2024-03-08T06:04:02.524598+0000","last_became_peered":"2024-03-08T06:04:02.524598+0000","last_unstale":"2024-03-08T06:04:03.553855+0000","last_undegraded":"2024-03-08T06:04:03.553855+0000","last_fullsized":"2024-03-08T06:04:03.553855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516807+0000","last_change":"2024-03-08T06:04:01.557363+0000","last_active":"2024-03-08T06:04:03.516807+0000","last_peered":"2024-03-08T06:04:03.516807+0000","last_clean":"2024-03-08T06:04:03.516807+0000","last_became_active":"2024-03-08T06:04:01.556516+0000","last_became_peered":"2024-03-08T06:04:01.556516+0000","last_unstale":"2024-03-08T06:04:03.516807+0000","last_undegraded":"2024-03-08T06:04:03.516807+0000","last_fullsized":"2024-03-08T06:04:03.516807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118240+0000","last_change":"2024-03-08T06:04:02.554738+0000","last_active":"2024-03-08T06:04:04.118240+0000","last_peered":"2024-03-08T06:04:04.118240+0000","last_clean":"2024-03-08T06:04:04.118240+0000","last_became_active":"2024-03-08T06:04:02.554446+0000","last_became_peered":"2024-03-08T06:04:02.554446+0000","last_unstale":"2024-03-08T06:04:04.118240+0000","last_undegraded":"2024-03-08T06:04:04.118240+0000","last_fullsized":"2024-03-08T06:04:04.118240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:51.718653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553561+0000","last_change":"2024-03-08T06:04:01.543134+0000","last_active":"2024-03-08T06:04:03.553561+0000","last_peered":"2024-03-08T06:04:03.553561+0000","last_clean":"2024-03-08T06:04:03.553561+0000","last_became_active":"2024-03-08T06:04:01.542768+0000","last_became_peered":"2024-03-08T06:04:01.542768+0000","last_unstale":"2024-03-08T06:04:03.553561+0000","last_undegraded":"2024-03-08T06:04:03.553561+0000","last_fullsized":"2024-03-08T06:04:03.553561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517531+0000","last_change":"2024-03-08T06:04:02.549139+0000","last_active":"2024-03-08T06:04:03.517531+0000","last_peered":"2024-03-08T06:04:03.517531+0000","last_clean":"2024-03-08T06:04:03.517531+0000","last_became_active":"2024-03-08T06:04:02.548765+0000","last_became_peered":"2024-03-08T06:04:02.548765+0000","last_unstale":"2024-03-08T06:04:03.517531+0000","last_undegraded":"2024-03-08T06:04:03.517531+0000","last_fullsized":"2024-03-08T06:04:03.517531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:41:43.397781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618664+0000","last_change":"2024-03-08T06:04:01.553438+0000","last_active":"2024-03-08T06:04:03.618664+0000","last_peered":"2024-03-08T06:04:03.618664+0000","last_clean":"2024-03-08T06:04:03.618664+0000","last_became_active":"2024-03-08T06:04:01.553120+0000","last_became_peered":"2024-03-08T06:04:01.553120+0000","last_unstale":"2024-03-08T06:04:03.618664+0000","last_undegraded":"2024-03-08T06:04:03.618664+0000","last_fullsized":"2024-03-08T06:04:03.618664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:05:58.801355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553556+0000","last_change":"2024-03-08T06:04:02.523565+0000","last_active":"2024-03-08T06:04:03.553556+0000","last_peered":"2024-03-08T06:04:03.553556+0000","last_clean":"2024-03-08T06:04:03.553556+0000","last_became_active":"2024-03-08T06:04:02.522603+0000","last_became_peered":"2024-03-08T06:04:02.522603+0000","last_unstale":"2024-03-08T06:04:03.553556+0000","last_undegraded":"2024-03-08T06:04:03.553556+0000","last_fullsized":"2024-03-08T06:04:03.553556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.557358+0000","last_change":"2024-03-08T06:04:01.559721+0000","last_active":"2024-03-08T06:04:03.557358+0000","last_peered":"2024-03-08T06:04:03.557358+0000","last_clean":"2024-03-08T06:04:03.557358+0000","last_became_active":"2024-03-08T06:04:01.559454+0000","last_became_peered":"2024-03-08T06:04:01.559454+0000","last_unstale":"2024-03-08T06:04:03.557358+0000","last_undegraded":"2024-03-08T06:04:03.557358+0000","last_fullsized":"2024-03-08T06:04:03.557358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512336+0000","last_change":"2024-03-08T06:04:02.555474+0000","last_active":"2024-03-08T06:04:03.512336+0000","last_peered":"2024-03-08T06:04:03.512336+0000","last_clean":"2024-03-08T06:04:03.512336+0000","last_became_active":"2024-03-08T06:04:02.551802+0000","last_became_peered":"2024-03-08T06:04:02.551802+0000","last_unstale":"2024-03-08T06:04:03.512336+0000","last_undegraded":"2024-03-08T06:04:03.512336+0000","last_fullsized":"2024-03-08T06:04:03.512336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559860+0000","last_change":"2024-03-08T06:04:01.559130+0000","last_active":"2024-03-08T06:04:03.559860+0000","last_peered":"2024-03-08T06:04:03.559860+0000","last_clean":"2024-03-08T06:04:03.559860+0000","last_became_active":"2024-03-08T06:04:01.558748+0000","last_became_peered":"2024-03-08T06:04:01.558748+0000","last_unstale":"2024-03-08T06:04:03.559860+0000","last_undegraded":"2024-03-08T06:04:03.559860+0000","last_fullsized":"2024-03-08T06:04:03.559860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.550035+0000","last_change":"2024-03-08T06:04:02.544456+0000","last_active":"2024-03-08T06:04:02.550035+0000","last_peered":"2024-03-08T06:04:02.550035+0000","last_clean":"2024-03-08T06:04:02.550035+0000","last_became_active":"2024-03-08T06:04:02.543887+0000","last_became_peered":"2024-03-08T06:04:02.543887+0000","last_unstale":"2024-03-08T06:04:02.550035+0000","last_undegraded":"2024-03-08T06:04:02.550035+0000","last_fullsized":"2024-03-08T06:04:02.550035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:15.104439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617019+0000","last_change":"2024-03-08T06:04:01.552997+0000","last_active":"2024-03-08T06:04:03.617019+0000","last_peered":"2024-03-08T06:04:03.617019+0000","last_clean":"2024-03-08T06:04:03.617019+0000","last_became_active":"2024-03-08T06:04:01.552617+0000","last_became_peered":"2024-03-08T06:04:01.552617+0000","last_unstale":"2024-03-08T06:04:03.617019+0000","last_undegraded":"2024-03-08T06:04:03.617019+0000","last_fullsized":"2024-03-08T06:04:03.617019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118081+0000","last_change":"2024-03-08T06:04:02.554162+0000","last_active":"2024-03-08T06:04:04.118081+0000","last_peered":"2024-03-08T06:04:04.118081+0000","last_clean":"2024-03-08T06:04:04.118081+0000","last_became_active":"2024-03-08T06:04:02.553850+0000","last_became_peered":"2024-03-08T06:04:02.553850+0000","last_unstale":"2024-03-08T06:04:04.118081+0000","last_undegraded":"2024-03-08T06:04:04.118081+0000","last_fullsized":"2024-03-08T06:04:04.118081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:09.964322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553380+0000","last_change":"2024-03-08T06:04:01.547872+0000","last_active":"2024-03-08T06:04:03.553380+0000","last_peered":"2024-03-08T06:04:03.553380+0000","last_clean":"2024-03-08T06:04:03.553380+0000","last_became_active":"2024-03-08T06:04:01.547553+0000","last_became_peered":"2024-03-08T06:04:01.547553+0000","last_unstale":"2024-03-08T06:04:03.553380+0000","last_undegraded":"2024-03-08T06:04:03.553380+0000","last_fullsized":"2024-03-08T06:04:03.553380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511556+0000","last_change":"2024-03-08T06:04:02.546841+0000","last_active":"2024-03-08T06:04:03.511556+0000","last_peered":"2024-03-08T06:04:03.511556+0000","last_clean":"2024-03-08T06:04:03.511556+0000","last_became_active":"2024-03-08T06:04:02.543164+0000","last_became_peered":"2024-03-08T06:04:02.543164+0000","last_unstale":"2024-03-08T06:04:03.511556+0000","last_undegraded":"2024-03-08T06:04:03.511556+0000","last_fullsized":"2024-03-08T06:04:03.511556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553503+0000","last_change":"2024-03-08T06:04:01.543186+0000","last_active":"2024-03-08T06:04:03.553503+0000","last_peered":"2024-03-08T06:04:03.553503+0000","last_clean":"2024-03-08T06:04:03.553503+0000","last_became_active":"2024-03-08T06:04:01.542185+0000","last_became_peered":"2024-03-08T06:04:01.542185+0000","last_unstale":"2024-03-08T06:04:03.553503+0000","last_undegraded":"2024-03-08T06:04:03.553503+0000","last_fullsized":"2024-03-08T06:04:03.553503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511983+0000","last_change":"2024-03-08T06:04:02.552865+0000","last_active":"2024-03-08T06:04:03.511983+0000","last_peered":"2024-03-08T06:04:03.511983+0000","last_clean":"2024-03-08T06:04:03.511983+0000","last_became_active":"2024-03-08T06:04:02.552563+0000","last_became_peered":"2024-03-08T06:04:02.552563+0000","last_unstale":"2024-03-08T06:04:03.511983+0000","last_undegraded":"2024-03-08T06:04:03.511983+0000","last_fullsized":"2024-03-08T06:04:03.511983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618134+0000","last_change":"2024-03-08T06:04:01.560452+0000","last_active":"2024-03-08T06:04:03.618134+0000","last_peered":"2024-03-08T06:04:03.618134+0000","last_clean":"2024-03-08T06:04:03.618134+0000","last_became_active":"2024-03-08T06:04:01.559883+0000","last_became_peered":"2024-03-08T06:04:01.559883+0000","last_unstale":"2024-03-08T06:04:03.618134+0000","last_undegraded":"2024-03-08T06:04:03.618134+0000","last_fullsized":"2024-03-08T06:04:03.618134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:50.471516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559002+0000","last_change":"2024-03-08T06:04:02.562767+0000","last_active":"2024-03-08T06:04:03.559002+0000","last_peered":"2024-03-08T06:04:03.559002+0000","last_clean":"2024-03-08T06:04:03.559002+0000","last_became_active":"2024-03-08T06:04:02.562425+0000","last_became_peered":"2024-03-08T06:04:02.562425+0000","last_unstale":"2024-03-08T06:04:03.559002+0000","last_undegraded":"2024-03-08T06:04:03.559002+0000","last_fullsized":"2024-03-08T06:04:03.559002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517395+0000","last_change":"2024-03-08T06:04:02.559515+0000","last_active":"2024-03-08T06:04:03.517395+0000","last_peered":"2024-03-08T06:04:03.517395+0000","last_clean":"2024-03-08T06:04:03.517395+0000","last_became_active":"2024-03-08T06:04:02.559082+0000","last_became_peered":"2024-03-08T06:04:02.559082+0000","last_unstale":"2024-03-08T06:04:03.517395+0000","last_undegraded":"2024-03-08T06:04:03.517395+0000","last_fullsized":"2024-03-08T06:04:03.517395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:42:12.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617836+0000","last_change":"2024-03-08T06:04:01.560587+0000","last_active":"2024-03-08T06:04:03.617836+0000","last_peered":"2024-03-08T06:04:03.617836+0000","last_clean":"2024-03-08T06:04:03.617836+0000","last_became_active":"2024-03-08T06:04:01.560275+0000","last_became_peered":"2024-03-08T06:04:01.560275+0000","last_unstale":"2024-03-08T06:04:03.617836+0000","last_undegraded":"2024-03-08T06:04:03.617836+0000","last_fullsized":"2024-03-08T06:04:03.617836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516385+0000","last_change":"2024-03-08T06:04:02.559339+0000","last_active":"2024-03-08T06:04:03.516385+0000","last_peered":"2024-03-08T06:04:03.516385+0000","last_clean":"2024-03-08T06:04:03.516385+0000","last_became_active":"2024-03-08T06:04:02.559023+0000","last_became_peered":"2024-03-08T06:04:02.559023+0000","last_unstale":"2024-03-08T06:04:03.516385+0000","last_undegraded":"2024-03-08T06:04:03.516385+0000","last_fullsized":"2024-03-08T06:04:03.516385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618371+0000","last_change":"2024-03-08T06:04:01.560256+0000","last_active":"2024-03-08T06:04:03.618371+0000","last_peered":"2024-03-08T06:04:03.618371+0000","last_clean":"2024-03-08T06:04:03.618371+0000","last_became_active":"2024-03-08T06:04:01.559917+0000","last_became_peered":"2024-03-08T06:04:01.559917+0000","last_unstale":"2024-03-08T06:04:03.618371+0000","last_undegraded":"2024-03-08T06:04:03.618371+0000","last_fullsized":"2024-03-08T06:04:03.618371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:55.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.564684+0000","last_change":"2024-03-08T06:04:02.551405+0000","last_active":"2024-03-08T06:04:02.564684+0000","last_peered":"2024-03-08T06:04:02.564684+0000","last_clean":"2024-03-08T06:04:02.564684+0000","last_became_active":"2024-03-08T06:04:02.550283+0000","last_became_peered":"2024-03-08T06:04:02.550283+0000","last_unstale":"2024-03-08T06:04:02.564684+0000","last_undegraded":"2024-03-08T06:04:02.564684+0000","last_fullsized":"2024-03-08T06:04:02.564684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:09.964322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512142+0000","last_change":"2024-03-08T06:04:01.540725+0000","last_active":"2024-03-08T06:04:03.512142+0000","last_peered":"2024-03-08T06:04:03.512142+0000","last_clean":"2024-03-08T06:04:03.512142+0000","last_became_active":"2024-03-08T06:04:01.540349+0000","last_became_peered":"2024-03-08T06:04:01.540349+0000","last_unstale":"2024-03-08T06:04:03.512142+0000","last_undegraded":"2024-03-08T06:04:03.512142+0000","last_fullsized":"2024-03-08T06:04:03.512142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512201+0000","last_change":"2024-03-08T06:04:02.554282+0000","last_active":"2024-03-08T06:04:03.512201+0000","last_peered":"2024-03-08T06:04:03.512201+0000","last_clean":"2024-03-08T06:04:03.512201+0000","last_became_active":"2024-03-08T06:04:02.551101+0000","last_became_peered":"2024-03-08T06:04:02.551101+0000","last_unstale":"2024-03-08T06:04:03.512201+0000","last_undegraded":"2024-03-08T06:04:03.512201+0000","last_fullsized":"2024-03-08T06:04:03.512201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.573670+0000","last_change":"2024-03-08T06:04:01.555646+0000","last_active":"2024-03-08T06:04:02.573670+0000","last_peered":"2024-03-08T06:04:02.573670+0000","last_clean":"2024-03-08T06:04:02.573670+0000","last_became_active":"2024-03-08T06:04:01.555404+0000","last_became_peered":"2024-03-08T06:04:01.555404+0000","last_unstale":"2024-03-08T06:04:02.573670+0000","last_undegraded":"2024-03-08T06:04:02.573670+0000","last_fullsized":"2024-03-08T06:04:02.573670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:52:36.740849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":69431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":69305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":811008,"data_stored":660311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:04:07.003 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:04:07.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.258+0000 7f63e6800640 1 Processor -- start 2024-03-08T06:04:07.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.258+0000 7f63e6800640 1 -- start start 2024-03-08T06:04:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.260+0000 7f63e6800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f63e006fb70 0x7f63e00b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.260+0000 7f63e6800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e00701e0 con 0x7f63e00b5c60 2024-03-08T06:04:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.260+0000 7f63e6800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e00704c0 con 0x7f63e00afb80 2024-03-08T06:04:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.260+0000 7f63e6800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e0151e90 con 0x7f63e006fb70 2024-03-08T06:04:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.260+0000 7f63e4a00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f63e00b5c60 0x7f63e006f460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60796/0 (socket says 172.21.15.40:60796) 2024-03-08T06:04:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.260+0000 7f63e4a00640 1 -- 172.21.15.40:0/2544127403 learned_addr learned my addr 172.21.15.40:0/2544127403 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.260+0000 7f63e5e00640 1 --2- 172.21.15.40:0/2544127403 >> v2:172.21.15.53:3300/0 conn(0x7f63e006fb70 0x7f63e00b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257999437 0 0) 0x7f63e00701e0 con 0x7f63e00b5c60 2024-03-08T06:04:07.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c8003520 con 0x7f63e00b5c60 2024-03-08T06:04:07.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 358924239 0 0) 0x7f63c8003520 con 0x7f63e00b5c60 2024-03-08T06:04:07.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 >> v2:172.21.15.53:3300/0 conn(0x7f63e006fb70 msgr2=0x7f63e00b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:07.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 --2- 172.21.15.40:0/2544127403 >> v2:172.21.15.53:3300/0 conn(0x7f63e006fb70 0x7f63e00b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 >> v1:172.21.15.53:6789/0 conn(0x7f63e00afb80 legacy=0x7f63e00add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:07.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e00b3990 con 0x7f63e00b5c60 2024-03-08T06:04:07.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f63cc001670 con 0x7f63e00b5c60 2024-03-08T06:04:07.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 >> v1:172.21.15.40:6789/0 conn(0x7f63e00b5c60 legacy=0x7f63e006f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f63c80039b0 0x7f63c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63c8004800 0x7f63c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c80051c0 0x7f63c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c800c030 con 0x7f63c80051c0 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c800c1e0 con 0x7f63c8004800 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c800c3e0 con 0x7f63c80039b0 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.261+0000 7f63e5e00640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c80051c0 0x7f63c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.262+0000 7f63e4a00640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63c8004800 0x7f63c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.262+0000 7f63e5400640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f63c80039b0 0x7f63c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.262+0000 7f63e5e00640 1 -- 172.21.15.40:0/2544127403 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f63c80039b0 msgr2=0x7f63c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.262+0000 7f63e5e00640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f63c80039b0 0x7f63c8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.262+0000 7f63e5e00640 1 -- 172.21.15.40:0/2544127403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63c8004800 msgr2=0x7f63c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.262+0000 7f63e5e00640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63c8004800 0x7f63c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.262+0000 7f63e5e00640 1 -- 172.21.15.40:0/2544127403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c8003520 con 0x7f63c80051c0 2024-03-08T06:04:07.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.262+0000 7f63e4a00640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63c8004800 0x7f63c8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:07.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.263+0000 7f63e5e00640 1 -- 172.21.15.40:0/2544127403 reap_dead start 2024-03-08T06:04:07.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.263+0000 7f63e5e00640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c80051c0 0x7f63c800b9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009ac0 tx=0x7f63c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90538efb7a5466c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:07.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.263+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8009690 con 0x7f63c80051c0 2024-03-08T06:04:07.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.263+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d8004020 con 0x7f63c80051c0 2024-03-08T06:04:07.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.263+0000 7f63dfe00640 1 -- 172.21.15.40:0/2544127403 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8004410 con 0x7f63c80051c0 2024-03-08T06:04:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.264+0000 7f63e6800640 1 -- 172.21.15.40:0/2544127403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c80051c0 msgr2=0x7f63c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.264+0000 7f63e6800640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c80051c0 0x7f63c800b9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009ac0 tx=0x7f63c8005830 comp rx=0 tx=0).stop 2024-03-08T06:04:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.264+0000 7f63e6800640 1 -- 172.21.15.40:0/2544127403 shutdown_connections 2024-03-08T06:04:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.264+0000 7f63e6800640 1 --2- 172.21.15.40:0/2544127403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c80051c0 0x7f63c800b9c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.264+0000 7f63e6800640 1 -- 172.21.15.40:0/2544127403 >> 172.21.15.40:0/2544127403 conn(0x7f63e0099a70 msgr2=0x7f63e0099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.264+0000 7f63e6800640 1 -- 172.21.15.40:0/2544127403 shutdown_connections 2024-03-08T06:04:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.264+0000 7f63e6800640 1 -- 172.21.15.40:0/2544127403 wait complete. 2024-03-08T06:04:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.265+0000 7f63e6800640 1 Processor -- start 2024-03-08T06:04:07.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.266+0000 7f63e6800640 1 -- start start 2024-03-08T06:04:07.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.266+0000 7f63e6800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f63c80051c0 0x7f63e00b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.266+0000 7f63e6800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e014b400 0x7f63e01468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e6800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63e0159990 0x7f63e0159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e6800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00aff60 con 0x7f63e0159990 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e6800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00b3740 con 0x7f63e014b400 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e6800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00b60c0 con 0x7f63c80051c0 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e5e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63e0159990 0x7f63e0159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e5e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63e0159990 0x7f63e0159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45542/0 (socket says 172.21.15.40:45542) 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e5e00640 1 -- 172.21.15.40:0/3254288584 learned_addr learned my addr 172.21.15.40:0/3254288584 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e4a00640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e014b400 0x7f63e01468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e5400640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f63c80051c0 0x7f63e00b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e5e00640 1 -- 172.21.15.40:0/3254288584 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f63c80051c0 msgr2=0x7f63e00b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e5e00640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f63c80051c0 0x7f63e00b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e5e00640 1 -- 172.21.15.40:0/3254288584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e014b400 msgr2=0x7f63e01468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e5e00640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e014b400 0x7f63e01468e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e5e00640 1 -- 172.21.15.40:0/3254288584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e014b880 con 0x7f63e0159990 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e5400640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f63c80051c0 0x7f63e00b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:07.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.267+0000 7f63e4a00640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e014b400 0x7f63e01468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:07.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.268+0000 7f63e5e00640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63e0159990 0x7f63e0159e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009bb0 tx=0x7f63d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:07.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.268+0000 7f63dd600640 1 -- 172.21.15.40:0/3254288584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8009690 con 0x7f63e0159990 2024-03-08T06:04:07.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.268+0000 7f63dd600640 1 -- 172.21.15.40:0/3254288584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d802cc50 con 0x7f63e0159990 2024-03-08T06:04:07.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.268+0000 7f63dd600640 1 -- 172.21.15.40:0/3254288584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8003610 con 0x7f63e0159990 2024-03-08T06:04:07.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.268+0000 7f63e6800640 1 -- 172.21.15.40:0/3254288584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e014dc30 con 0x7f63e0159990 2024-03-08T06:04:07.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.268+0000 7f63e6800640 1 -- 172.21.15.40:0/3254288584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e014def0 con 0x7f63e0159990 2024-03-08T06:04:07.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.269+0000 7f63e6800640 1 -- 172.21.15.40:0/3254288584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e015a350 con 0x7f63e0159990 2024-03-08T06:04:07.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.270+0000 7f63dd600640 1 -- 172.21.15.40:0/3254288584 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63d80037b0 con 0x7f63e0159990 2024-03-08T06:04:07.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.271+0000 7f63dd600640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f63c006b500 0x7f63c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.271+0000 7f63dd600640 1 -- 172.21.15.40:0/3254288584 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63d80aab30 con 0x7f63e0159990 2024-03-08T06:04:07.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.271+0000 7f63e5400640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f63c006b500 0x7f63c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.272+0000 7f63e5400640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f63c006b500 0x7f63c006d9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63d4001ce0 tx=0x7f63d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:07.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.276+0000 7f63dd600640 1 -- 172.21.15.40:0/3254288584 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63d8046030 con 0x7f63e0159990 2024-03-08T06:04:07.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.471+0000 7f63e6800640 1 -- 172.21.15.40:0/3254288584 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63e0075100 con 0x7f63c006b500 2024-03-08T06:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.478+0000 7f63dd600640 1 -- 172.21.15.40:0/3254288584 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251936 (secure 0 0 0) 0x7f63e0075100 con 0x7f63c006b500 2024-03-08T06:04:07.479 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:07.483 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-03-08T06:04:07.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 -- 172.21.15.40:0/3254288584 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f63c006b500 msgr2=0x7f63c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:07.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f63c006b500 0x7f63c006d9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63d4001ce0 tx=0x7f63d4009040 comp rx=0 tx=0).stop 2024-03-08T06:04:07.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 -- 172.21.15.40:0/3254288584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63e0159990 msgr2=0x7f63e0159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:07.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63e0159990 0x7f63e0159e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009bb0 tx=0x7f63d8004730 comp rx=0 tx=0).stop 2024-03-08T06:04:07.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 -- 172.21.15.40:0/3254288584 shutdown_connections 2024-03-08T06:04:07.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f63c006b500 0x7f63c006d9c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63e0159990 0x7f63e0159e10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63e014b400 0x7f63e01468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 --2- 172.21.15.40:0/3254288584 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f63c80051c0 0x7f63e00b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 -- 172.21.15.40:0/3254288584 >> 172.21.15.40:0/3254288584 conn(0x7f63e0099a70 msgr2=0x7f63e015c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:07.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 -- 172.21.15.40:0/3254288584 shutdown_connections 2024-03-08T06:04:07.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.483+0000 7f63e6800640 1 -- 172.21.15.40:0/3254288584 wait complete. 2024-03-08T06:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-03-08T06:04:05.483554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236112,"kb_used_data":2884,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738608,"statfs":{"total":773094113280,"available":772852334592,"internally_reserved":0,"allocated":2953216,"data_stored":1771496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.726509"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618708+0000","last_change":"2024-03-08T06:04:02.552950+0000","last_active":"2024-03-08T06:04:03.618708+0000","last_peered":"2024-03-08T06:04:03.618708+0000","last_clean":"2024-03-08T06:04:03.618708+0000","last_became_active":"2024-03-08T06:04:02.552600+0000","last_became_peered":"2024-03-08T06:04:02.552600+0000","last_unstale":"2024-03-08T06:04:03.618708+0000","last_undegraded":"2024-03-08T06:04:03.618708+0000","last_fullsized":"2024-03-08T06:04:03.618708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:13:36.393811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553892+0000","last_change":"2024-03-08T06:04:02.547475+0000","last_active":"2024-03-08T06:04:03.553892+0000","last_peered":"2024-03-08T06:04:03.553892+0000","last_clean":"2024-03-08T06:04:03.553892+0000","last_became_active":"2024-03-08T06:04:02.545953+0000","last_became_peered":"2024-03-08T06:04:02.545953+0000","last_unstale":"2024-03-08T06:04:03.553892+0000","last_undegraded":"2024-03-08T06:04:03.553892+0000","last_fullsized":"2024-03-08T06:04:03.553892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.550165+0000","last_change":"2024-03-08T06:04:02.544781+0000","last_active":"2024-03-08T06:04:02.550165+0000","last_peered":"2024-03-08T06:04:02.550165+0000","last_clean":"2024-03-08T06:04:02.550165+0000","last_became_active":"2024-03-08T06:04:02.543995+0000","last_became_peered":"2024-03-08T06:04:02.543995+0000","last_unstale":"2024-03-08T06:04:02.550165+0000","last_undegraded":"2024-03-08T06:04:02.550165+0000","last_fullsized":"2024-03-08T06:04:02.550165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:59.913059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515851+0000","last_change":"2024-03-08T06:04:02.551644+0000","last_active":"2024-03-08T06:04:03.515851+0000","last_peered":"2024-03-08T06:04:03.515851+0000","last_clean":"2024-03-08T06:04:03.515851+0000","last_became_active":"2024-03-08T06:04:02.550930+0000","last_became_peered":"2024-03-08T06:04:02.550930+0000","last_unstale":"2024-03-08T06:04:03.515851+0000","last_undegraded":"2024-03-08T06:04:03.515851+0000","last_fullsized":"2024-03-08T06:04:03.515851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:23:55.655734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617514+0000","last_change":"2024-03-08T06:04:02.545560+0000","last_active":"2024-03-08T06:04:03.617514+0000","last_peered":"2024-03-08T06:04:03.617514+0000","last_clean":"2024-03-08T06:04:03.617514+0000","last_became_active":"2024-03-08T06:04:02.543855+0000","last_became_peered":"2024-03-08T06:04:02.543855+0000","last_unstale":"2024-03-08T06:04:03.617514+0000","last_undegraded":"2024-03-08T06:04:03.617514+0000","last_fullsized":"2024-03-08T06:04:03.617514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515916+0000","last_change":"2024-03-08T06:04:02.550172+0000","last_active":"2024-03-08T06:04:03.515916+0000","last_peered":"2024-03-08T06:04:03.515916+0000","last_clean":"2024-03-08T06:04:03.515916+0000","last_became_active":"2024-03-08T06:04:02.549807+0000","last_became_peered":"2024-03-08T06:04:02.549807+0000","last_unstale":"2024-03-08T06:04:03.515916+0000","last_undegraded":"2024-03-08T06:04:03.515916+0000","last_fullsized":"2024-03-08T06:04:03.515916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617257+0000","last_change":"2024-03-08T06:04:02.547691+0000","last_active":"2024-03-08T06:04:03.617257+0000","last_peered":"2024-03-08T06:04:03.617257+0000","last_clean":"2024-03-08T06:04:03.617257+0000","last_became_active":"2024-03-08T06:04:02.547325+0000","last_became_peered":"2024-03-08T06:04:02.547325+0000","last_unstale":"2024-03-08T06:04:03.617257+0000","last_undegraded":"2024-03-08T06:04:03.617257+0000","last_fullsized":"2024-03-08T06:04:03.617257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511377+0000","last_change":"2024-03-08T06:04:02.553110+0000","last_active":"2024-03-08T06:04:03.511377+0000","last_peered":"2024-03-08T06:04:03.511377+0000","last_clean":"2024-03-08T06:04:03.511377+0000","last_became_active":"2024-03-08T06:04:02.552635+0000","last_became_peered":"2024-03-08T06:04:02.552635+0000","last_unstale":"2024-03-08T06:04:03.511377+0000","last_undegraded":"2024-03-08T06:04:03.511377+0000","last_fullsized":"2024-03-08T06:04:03.511377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618751+0000","last_change":"2024-03-08T06:04:02.570801+0000","last_active":"2024-03-08T06:04:03.618751+0000","last_peered":"2024-03-08T06:04:03.618751+0000","last_clean":"2024-03-08T06:04:03.618751+0000","last_became_active":"2024-03-08T06:04:02.570558+0000","last_became_peered":"2024-03-08T06:04:02.570558+0000","last_unstale":"2024-03-08T06:04:03.618751+0000","last_undegraded":"2024-03-08T06:04:03.618751+0000","last_fullsized":"2024-03-08T06:04:03.618751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:07:57.853863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559891+0000","last_change":"2024-03-08T06:04:02.562132+0000","last_active":"2024-03-08T06:04:03.559891+0000","last_peered":"2024-03-08T06:04:03.559891+0000","last_clean":"2024-03-08T06:04:03.559891+0000","last_became_active":"2024-03-08T06:04:02.561851+0000","last_became_peered":"2024-03-08T06:04:02.561851+0000","last_unstale":"2024-03-08T06:04:03.559891+0000","last_undegraded":"2024-03-08T06:04:03.559891+0000","last_fullsized":"2024-03-08T06:04:03.559891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559579+0000","last_change":"2024-03-08T06:04:02.546846+0000","last_active":"2024-03-08T06:04:03.559579+0000","last_peered":"2024-03-08T06:04:03.559579+0000","last_clean":"2024-03-08T06:04:03.559579+0000","last_became_active":"2024-03-08T06:04:02.546522+0000","last_became_peered":"2024-03-08T06:04:02.546522+0000","last_unstale":"2024-03-08T06:04:03.559579+0000","last_undegraded":"2024-03-08T06:04:03.559579+0000","last_fullsized":"2024-03-08T06:04:03.559579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118422+0000","last_change":"2024-03-08T06:04:02.555942+0000","last_active":"2024-03-08T06:04:04.118422+0000","last_peered":"2024-03-08T06:04:04.118422+0000","last_clean":"2024-03-08T06:04:04.118422+0000","last_became_active":"2024-03-08T06:04:02.555651+0000","last_became_peered":"2024-03-08T06:04:02.555651+0000","last_unstale":"2024-03-08T06:04:04.118422+0000","last_undegraded":"2024-03-08T06:04:04.118422+0000","last_fullsized":"2024-03-08T06:04:04.118422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:57.032278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511828+0000","last_change":"2024-03-08T06:04:02.551969+0000","last_active":"2024-03-08T06:04:03.511828+0000","last_peered":"2024-03-08T06:04:03.511828+0000","last_clean":"2024-03-08T06:04:03.511828+0000","last_became_active":"2024-03-08T06:04:02.551664+0000","last_became_peered":"2024-03-08T06:04:02.551664+0000","last_unstale":"2024-03-08T06:04:03.511828+0000","last_undegraded":"2024-03-08T06:04:03.511828+0000","last_fullsized":"2024-03-08T06:04:03.511828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.547045+0000","last_change":"2024-03-08T06:04:02.542608+0000","last_active":"2024-03-08T06:04:02.547045+0000","last_peered":"2024-03-08T06:04:02.547045+0000","last_clean":"2024-03-08T06:04:02.547045+0000","last_became_active":"2024-03-08T06:04:02.542318+0000","last_became_peered":"2024-03-08T06:04:02.542318+0000","last_unstale":"2024-03-08T06:04:02.547045+0000","last_undegraded":"2024-03-08T06:04:02.547045+0000","last_fullsized":"2024-03-08T06:04:02.547045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:33.768388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553245+0000","last_change":"2024-03-08T06:04:02.548664+0000","last_active":"2024-03-08T06:04:03.553245+0000","last_peered":"2024-03-08T06:04:03.553245+0000","last_clean":"2024-03-08T06:04:03.553245+0000","last_became_active":"2024-03-08T06:04:02.547601+0000","last_became_peered":"2024-03-08T06:04:02.547601+0000","last_unstale":"2024-03-08T06:04:03.553245+0000","last_undegraded":"2024-03-08T06:04:03.553245+0000","last_fullsized":"2024-03-08T06:04:03.553245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515232+0000","last_change":"2024-03-08T06:04:02.551648+0000","last_active":"2024-03-08T06:04:03.515232+0000","last_peered":"2024-03-08T06:04:03.515232+0000","last_clean":"2024-03-08T06:04:03.515232+0000","last_became_active":"2024-03-08T06:04:02.551290+0000","last_became_peered":"2024-03-08T06:04:02.551290+0000","last_unstale":"2024-03-08T06:04:03.515232+0000","last_undegraded":"2024-03-08T06:04:03.515232+0000","last_fullsized":"2024-03-08T06:04:03.515232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.518022+0000","last_change":"2024-03-08T06:04:02.556704+0000","last_active":"2024-03-08T06:04:03.518022+0000","last_peered":"2024-03-08T06:04:03.518022+0000","last_clean":"2024-03-08T06:04:03.518022+0000","last_became_active":"2024-03-08T06:04:02.556429+0000","last_became_peered":"2024-03-08T06:04:02.556429+0000","last_unstale":"2024-03-08T06:04:03.518022+0000","last_undegraded":"2024-03-08T06:04:03.518022+0000","last_fullsized":"2024-03-08T06:04:03.518022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:13:36.393811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.543918+0000","last_change":"2024-03-08T06:04:02.539066+0000","last_active":"2024-03-08T06:04:02.543918+0000","last_peered":"2024-03-08T06:04:02.543918+0000","last_clean":"2024-03-08T06:04:02.543918+0000","last_became_active":"2024-03-08T06:04:02.538745+0000","last_became_peered":"2024-03-08T06:04:02.538745+0000","last_unstale":"2024-03-08T06:04:02.543918+0000","last_undegraded":"2024-03-08T06:04:02.543918+0000","last_fullsized":"2024-03-08T06:04:02.543918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:52.535460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618590+0000","last_change":"2024-03-08T06:04:02.551987+0000","last_active":"2024-03-08T06:04:03.618590+0000","last_peered":"2024-03-08T06:04:03.618590+0000","last_clean":"2024-03-08T06:04:03.618590+0000","last_became_active":"2024-03-08T06:04:02.551425+0000","last_became_peered":"2024-03-08T06:04:02.551425+0000","last_unstale":"2024-03-08T06:04:03.618590+0000","last_undegraded":"2024-03-08T06:04:03.618590+0000","last_fullsized":"2024-03-08T06:04:03.618590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:41:43.397781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559135+0000","last_change":"2024-03-08T06:04:02.559334+0000","last_active":"2024-03-08T06:04:03.559135+0000","last_peered":"2024-03-08T06:04:03.559135+0000","last_clean":"2024-03-08T06:04:03.559135+0000","last_became_active":"2024-03-08T06:04:02.558979+0000","last_became_peered":"2024-03-08T06:04:02.558979+0000","last_unstale":"2024-03-08T06:04:03.559135+0000","last_undegraded":"2024-03-08T06:04:03.559135+0000","last_fullsized":"2024-03-08T06:04:03.559135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.550247+0000","last_change":"2024-03-08T06:04:02.545858+0000","last_active":"2024-03-08T06:04:02.550247+0000","last_peered":"2024-03-08T06:04:02.550247+0000","last_clean":"2024-03-08T06:04:02.550247+0000","last_became_active":"2024-03-08T06:04:02.545518+0000","last_became_peered":"2024-03-08T06:04:02.545518+0000","last_unstale":"2024-03-08T06:04:02.550247+0000","last_undegraded":"2024-03-08T06:04:02.550247+0000","last_fullsized":"2024-03-08T06:04:02.550247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:19.592493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516005+0000","last_change":"2024-03-08T06:04:02.553462+0000","last_active":"2024-03-08T06:04:03.516005+0000","last_peered":"2024-03-08T06:04:03.516005+0000","last_clean":"2024-03-08T06:04:03.516005+0000","last_became_active":"2024-03-08T06:04:02.552688+0000","last_became_peered":"2024-03-08T06:04:02.552688+0000","last_unstale":"2024-03-08T06:04:03.516005+0000","last_undegraded":"2024-03-08T06:04:03.516005+0000","last_fullsized":"2024-03-08T06:04:03.516005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511690+0000","last_change":"2024-03-08T06:04:02.560361+0000","last_active":"2024-03-08T06:04:03.511690+0000","last_peered":"2024-03-08T06:04:03.511690+0000","last_clean":"2024-03-08T06:04:03.511690+0000","last_became_active":"2024-03-08T06:04:02.560083+0000","last_became_peered":"2024-03-08T06:04:02.560083+0000","last_unstale":"2024-03-08T06:04:03.511690+0000","last_undegraded":"2024-03-08T06:04:03.511690+0000","last_fullsized":"2024-03-08T06:04:03.511690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515403+0000","last_change":"2024-03-08T06:04:02.549163+0000","last_active":"2024-03-08T06:04:03.515403+0000","last_peered":"2024-03-08T06:04:03.515403+0000","last_clean":"2024-03-08T06:04:03.515403+0000","last_became_active":"2024-03-08T06:04:02.548715+0000","last_became_peered":"2024-03-08T06:04:02.548715+0000","last_unstale":"2024-03-08T06:04:03.515403+0000","last_undegraded":"2024-03-08T06:04:03.515403+0000","last_fullsized":"2024-03-08T06:04:03.515403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512314+0000","last_change":"2024-03-08T06:04:02.565662+0000","last_active":"2024-03-08T06:04:03.512314+0000","last_peered":"2024-03-08T06:04:03.512314+0000","last_clean":"2024-03-08T06:04:03.512314+0000","last_became_active":"2024-03-08T06:04:02.565344+0000","last_became_peered":"2024-03-08T06:04:02.565344+0000","last_unstale":"2024-03-08T06:04:03.512314+0000","last_undegraded":"2024-03-08T06:04:03.512314+0000","last_fullsized":"2024-03-08T06:04:03.512314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:23:55.655734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.560225+0000","last_change":"2024-03-08T06:04:02.545865+0000","last_active":"2024-03-08T06:04:03.560225+0000","last_peered":"2024-03-08T06:04:03.560225+0000","last_clean":"2024-03-08T06:04:03.560225+0000","last_became_active":"2024-03-08T06:04:02.543828+0000","last_became_peered":"2024-03-08T06:04:02.543828+0000","last_unstale":"2024-03-08T06:04:03.560225+0000","last_undegraded":"2024-03-08T06:04:03.560225+0000","last_fullsized":"2024-03-08T06:04:03.560225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553769+0000","last_change":"2024-03-08T06:04:02.544859+0000","last_active":"2024-03-08T06:04:03.553769+0000","last_peered":"2024-03-08T06:04:03.553769+0000","last_clean":"2024-03-08T06:04:03.553769+0000","last_became_active":"2024-03-08T06:04:02.544502+0000","last_became_peered":"2024-03-08T06:04:02.544502+0000","last_unstale":"2024-03-08T06:04:03.553769+0000","last_undegraded":"2024-03-08T06:04:03.553769+0000","last_fullsized":"2024-03-08T06:04:03.553769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512315+0000","last_change":"2024-03-08T06:04:02.563733+0000","last_active":"2024-03-08T06:04:03.512315+0000","last_peered":"2024-03-08T06:04:03.512315+0000","last_clean":"2024-03-08T06:04:03.512315+0000","last_became_active":"2024-03-08T06:04:02.563420+0000","last_became_peered":"2024-03-08T06:04:02.563420+0000","last_unstale":"2024-03-08T06:04:03.512315+0000","last_undegraded":"2024-03-08T06:04:03.512315+0000","last_fullsized":"2024-03-08T06:04:03.512315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.551336+0000","last_change":"2024-03-08T06:04:02.548150+0000","last_active":"2024-03-08T06:04:02.551336+0000","last_peered":"2024-03-08T06:04:02.551336+0000","last_clean":"2024-03-08T06:04:02.551336+0000","last_became_active":"2024-03-08T06:04:02.547846+0000","last_became_peered":"2024-03-08T06:04:02.547846+0000","last_unstale":"2024-03-08T06:04:02.551336+0000","last_undegraded":"2024-03-08T06:04:02.551336+0000","last_fullsized":"2024-03-08T06:04:02.551336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:52.535460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516515+0000","last_change":"2024-03-08T06:04:02.547204+0000","last_active":"2024-03-08T06:04:03.516515+0000","last_peered":"2024-03-08T06:04:03.516515+0000","last_clean":"2024-03-08T06:04:03.516515+0000","last_became_active":"2024-03-08T06:04:02.546850+0000","last_became_peered":"2024-03-08T06:04:02.546850+0000","last_unstale":"2024-03-08T06:04:03.516515+0000","last_undegraded":"2024-03-08T06:04:03.516515+0000","last_fullsized":"2024-03-08T06:04:03.516515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:32:53.947147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.559191+0000","last_change":"2024-03-08T06:04:02.553107+0000","last_active":"2024-03-08T06:04:02.559191+0000","last_peered":"2024-03-08T06:04:02.559191+0000","last_clean":"2024-03-08T06:04:02.559191+0000","last_became_active":"2024-03-08T06:04:02.552862+0000","last_became_peered":"2024-03-08T06:04:02.552862+0000","last_unstale":"2024-03-08T06:04:02.559191+0000","last_undegraded":"2024-03-08T06:04:02.559191+0000","last_fullsized":"2024-03-08T06:04:02.559191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:09.964322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559013+0000","last_change":"2024-03-08T06:04:02.544639+0000","last_active":"2024-03-08T06:04:03.559013+0000","last_peered":"2024-03-08T06:04:03.559013+0000","last_clean":"2024-03-08T06:04:03.559013+0000","last_became_active":"2024-03-08T06:04:02.544193+0000","last_became_peered":"2024-03-08T06:04:02.544193+0000","last_unstale":"2024-03-08T06:04:03.559013+0000","last_undegraded":"2024-03-08T06:04:03.559013+0000","last_fullsized":"2024-03-08T06:04:03.559013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.117892+0000","last_change":"2024-03-08T06:04:02.558328+0000","last_active":"2024-03-08T06:04:04.117892+0000","last_peered":"2024-03-08T06:04:04.117892+0000","last_clean":"2024-03-08T06:04:04.117892+0000","last_became_active":"2024-03-08T06:04:02.555015+0000","last_became_peered":"2024-03-08T06:04:02.555015+0000","last_unstale":"2024-03-08T06:04:04.117892+0000","last_undegraded":"2024-03-08T06:04:04.117892+0000","last_fullsized":"2024-03-08T06:04:04.117892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:52.535460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512099+0000","last_change":"2024-03-08T06:04:01.555471+0000","last_active":"2024-03-08T06:04:03.512099+0000","last_peered":"2024-03-08T06:04:03.512099+0000","last_clean":"2024-03-08T06:04:03.512099+0000","last_became_active":"2024-03-08T06:04:01.555171+0000","last_became_peered":"2024-03-08T06:04:01.555171+0000","last_unstale":"2024-03-08T06:04:03.512099+0000","last_undegraded":"2024-03-08T06:04:03.512099+0000","last_fullsized":"2024-03-08T06:04:03.512099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.566828+0000","last_change":"2024-03-08T06:04:02.563404+0000","last_active":"2024-03-08T06:04:02.566828+0000","last_peered":"2024-03-08T06:04:02.566828+0000","last_clean":"2024-03-08T06:04:02.566828+0000","last_became_active":"2024-03-08T06:04:02.558959+0000","last_became_peered":"2024-03-08T06:04:02.558959+0000","last_unstale":"2024-03-08T06:04:02.566828+0000","last_undegraded":"2024-03-08T06:04:02.566828+0000","last_fullsized":"2024-03-08T06:04:02.566828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:06:00.048492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559117+0000","last_change":"2024-03-08T06:04:01.549647+0000","last_active":"2024-03-08T06:04:03.559117+0000","last_peered":"2024-03-08T06:04:03.559117+0000","last_clean":"2024-03-08T06:04:03.559117+0000","last_became_active":"2024-03-08T06:04:01.549345+0000","last_became_peered":"2024-03-08T06:04:01.549345+0000","last_unstale":"2024-03-08T06:04:03.559117+0000","last_undegraded":"2024-03-08T06:04:03.559117+0000","last_fullsized":"2024-03-08T06:04:03.559117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511841+0000","last_change":"2024-03-08T06:04:01.551862+0000","last_active":"2024-03-08T06:04:03.511841+0000","last_peered":"2024-03-08T06:04:03.511841+0000","last_clean":"2024-03-08T06:04:03.511841+0000","last_became_active":"2024-03-08T06:04:01.551606+0000","last_became_peered":"2024-03-08T06:04:01.551606+0000","last_unstale":"2024-03-08T06:04:03.511841+0000","last_undegraded":"2024-03-08T06:04:03.511841+0000","last_fullsized":"2024-03-08T06:04:03.511841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:52:36.740849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512342+0000","last_change":"2024-03-08T06:04:02.548972+0000","last_active":"2024-03-08T06:04:03.512342+0000","last_peered":"2024-03-08T06:04:03.512342+0000","last_clean":"2024-03-08T06:04:03.512342+0000","last_became_active":"2024-03-08T06:04:02.548136+0000","last_became_peered":"2024-03-08T06:04:02.548136+0000","last_unstale":"2024-03-08T06:04:03.512342+0000","last_undegraded":"2024-03-08T06:04:03.512342+0000","last_fullsized":"2024-03-08T06:04:03.512342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516175+0000","last_change":"2024-03-08T06:04:01.556755+0000","last_active":"2024-03-08T06:04:03.516175+0000","last_peered":"2024-03-08T06:04:03.516175+0000","last_clean":"2024-03-08T06:04:03.516175+0000","last_became_active":"2024-03-08T06:04:01.555842+0000","last_became_peered":"2024-03-08T06:04:01.555842+0000","last_unstale":"2024-03-08T06:04:03.516175+0000","last_undegraded":"2024-03-08T06:04:03.516175+0000","last_fullsized":"2024-03-08T06:04:03.516175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553244+0000","last_change":"2024-03-08T06:04:02.549137+0000","last_active":"2024-03-08T06:04:03.553244+0000","last_peered":"2024-03-08T06:04:03.553244+0000","last_clean":"2024-03-08T06:04:03.553244+0000","last_became_active":"2024-03-08T06:04:02.547990+0000","last_became_peered":"2024-03-08T06:04:02.547990+0000","last_unstale":"2024-03-08T06:04:03.553244+0000","last_undegraded":"2024-03-08T06:04:03.553244+0000","last_fullsized":"2024-03-08T06:04:03.553244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559731+0000","last_change":"2024-03-08T06:04:01.555948+0000","last_active":"2024-03-08T06:04:03.559731+0000","last_peered":"2024-03-08T06:04:03.559731+0000","last_clean":"2024-03-08T06:04:03.559731+0000","last_became_active":"2024-03-08T06:04:01.555666+0000","last_became_peered":"2024-03-08T06:04:01.555666+0000","last_unstale":"2024-03-08T06:04:03.559731+0000","last_undegraded":"2024-03-08T06:04:03.559731+0000","last_fullsized":"2024-03-08T06:04:03.559731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517528+0000","last_change":"2024-03-08T06:04:02.558089+0000","last_active":"2024-03-08T06:04:03.517528+0000","last_peered":"2024-03-08T06:04:03.517528+0000","last_clean":"2024-03-08T06:04:03.517528+0000","last_became_active":"2024-03-08T06:04:02.557697+0000","last_became_peered":"2024-03-08T06:04:02.557697+0000","last_unstale":"2024-03-08T06:04:03.517528+0000","last_undegraded":"2024-03-08T06:04:03.517528+0000","last_fullsized":"2024-03-08T06:04:03.517528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:19:21.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517312+0000","last_change":"2024-03-08T06:04:01.559428+0000","last_active":"2024-03-08T06:04:03.517312+0000","last_peered":"2024-03-08T06:04:03.517312+0000","last_clean":"2024-03-08T06:04:03.517312+0000","last_became_active":"2024-03-08T06:04:01.559156+0000","last_became_peered":"2024-03-08T06:04:01.559156+0000","last_unstale":"2024-03-08T06:04:03.517312+0000","last_undegraded":"2024-03-08T06:04:03.517312+0000","last_fullsized":"2024-03-08T06:04:03.517312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.564662+0000","last_change":"2024-03-08T06:04:02.551412+0000","last_active":"2024-03-08T06:04:02.564662+0000","last_peered":"2024-03-08T06:04:02.564662+0000","last_clean":"2024-03-08T06:04:02.564662+0000","last_became_active":"2024-03-08T06:04:02.550080+0000","last_became_peered":"2024-03-08T06:04:02.550080+0000","last_unstale":"2024-03-08T06:04:02.564662+0000","last_undegraded":"2024-03-08T06:04:02.564662+0000","last_fullsized":"2024-03-08T06:04:02.564662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:51.718653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553697+0000","last_change":"2024-03-08T06:04:01.554654+0000","last_active":"2024-03-08T06:04:03.553697+0000","last_peered":"2024-03-08T06:04:03.553697+0000","last_clean":"2024-03-08T06:04:03.553697+0000","last_became_active":"2024-03-08T06:04:01.554238+0000","last_became_peered":"2024-03-08T06:04:01.554238+0000","last_unstale":"2024-03-08T06:04:03.553697+0000","last_undegraded":"2024-03-08T06:04:03.553697+0000","last_fullsized":"2024-03-08T06:04:03.553697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118544+0000","last_change":"2024-03-08T06:04:02.556415+0000","last_active":"2024-03-08T06:04:04.118544+0000","last_peered":"2024-03-08T06:04:04.118544+0000","last_clean":"2024-03-08T06:04:04.118544+0000","last_became_active":"2024-03-08T06:04:02.556055+0000","last_became_peered":"2024-03-08T06:04:02.556055+0000","last_unstale":"2024-03-08T06:04:04.118544+0000","last_undegraded":"2024-03-08T06:04:04.118544+0000","last_fullsized":"2024-03-08T06:04:04.118544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:06:00.048492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559371+0000","last_change":"2024-03-08T06:04:01.549872+0000","last_active":"2024-03-08T06:04:03.559371+0000","last_peered":"2024-03-08T06:04:03.559371+0000","last_clean":"2024-03-08T06:04:03.559371+0000","last_became_active":"2024-03-08T06:04:01.549313+0000","last_became_peered":"2024-03-08T06:04:01.549313+0000","last_unstale":"2024-03-08T06:04:03.559371+0000","last_undegraded":"2024-03-08T06:04:03.559371+0000","last_fullsized":"2024-03-08T06:04:03.559371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618023+0000","last_change":"2024-03-08T06:04:02.549539+0000","last_active":"2024-03-08T06:04:03.618023+0000","last_peered":"2024-03-08T06:04:03.618023+0000","last_clean":"2024-03-08T06:04:03.618023+0000","last_became_active":"2024-03-08T06:04:02.549245+0000","last_became_peered":"2024-03-08T06:04:02.549245+0000","last_unstale":"2024-03-08T06:04:03.618023+0000","last_undegraded":"2024-03-08T06:04:03.618023+0000","last_fullsized":"2024-03-08T06:04:03.618023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:33:11.525956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.574228+0000","last_change":"2024-03-08T06:04:01.542685+0000","last_active":"2024-03-08T06:04:02.574228+0000","last_peered":"2024-03-08T06:04:02.574228+0000","last_clean":"2024-03-08T06:04:02.574228+0000","last_became_active":"2024-03-08T06:04:01.542236+0000","last_became_peered":"2024-03-08T06:04:01.542236+0000","last_unstale":"2024-03-08T06:04:02.574228+0000","last_undegraded":"2024-03-08T06:04:02.574228+0000","last_fullsized":"2024-03-08T06:04:02.574228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:35:24.125408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.564748+0000","last_change":"2024-03-08T06:04:02.553210+0000","last_active":"2024-03-08T06:04:02.564748+0000","last_peered":"2024-03-08T06:04:02.564748+0000","last_clean":"2024-03-08T06:04:02.564748+0000","last_became_active":"2024-03-08T06:04:02.552889+0000","last_became_peered":"2024-03-08T06:04:02.552889+0000","last_unstale":"2024-03-08T06:04:02.564748+0000","last_undegraded":"2024-03-08T06:04:02.564748+0000","last_fullsized":"2024-03-08T06:04:02.564748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:57.032278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516402+0000","last_change":"2024-03-08T06:04:01.554506+0000","last_active":"2024-03-08T06:04:03.516402+0000","last_peered":"2024-03-08T06:04:03.516402+0000","last_clean":"2024-03-08T06:04:03.516402+0000","last_became_active":"2024-03-08T06:04:01.554224+0000","last_became_peered":"2024-03-08T06:04:01.554224+0000","last_unstale":"2024-03-08T06:04:03.516402+0000","last_undegraded":"2024-03-08T06:04:03.516402+0000","last_fullsized":"2024-03-08T06:04:03.516402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617833+0000","last_change":"2024-03-08T06:04:02.560754+0000","last_active":"2024-03-08T06:04:03.617833+0000","last_peered":"2024-03-08T06:04:03.617833+0000","last_clean":"2024-03-08T06:04:03.617833+0000","last_became_active":"2024-03-08T06:04:02.560482+0000","last_became_peered":"2024-03-08T06:04:02.560482+0000","last_unstale":"2024-03-08T06:04:03.617833+0000","last_undegraded":"2024-03-08T06:04:03.617833+0000","last_fullsized":"2024-03-08T06:04:03.617833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:32:53.947147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559130+0000","last_change":"2024-03-08T06:04:01.545961+0000","last_active":"2024-03-08T06:04:03.559130+0000","last_peered":"2024-03-08T06:04:03.559130+0000","last_clean":"2024-03-08T06:04:03.559130+0000","last_became_active":"2024-03-08T06:04:01.545579+0000","last_became_peered":"2024-03-08T06:04:01.545579+0000","last_unstale":"2024-03-08T06:04:03.559130+0000","last_undegraded":"2024-03-08T06:04:03.559130+0000","last_fullsized":"2024-03-08T06:04:03.559130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.558973+0000","last_change":"2024-03-08T06:04:02.544735+0000","last_active":"2024-03-08T06:04:03.558973+0000","last_peered":"2024-03-08T06:04:03.558973+0000","last_clean":"2024-03-08T06:04:03.558973+0000","last_became_active":"2024-03-08T06:04:02.544438+0000","last_became_peered":"2024-03-08T06:04:02.544438+0000","last_unstale":"2024-03-08T06:04:03.558973+0000","last_undegraded":"2024-03-08T06:04:03.558973+0000","last_fullsized":"2024-03-08T06:04:03.558973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.568401+0000","last_change":"2024-03-08T06:04:01.541060+0000","last_active":"2024-03-08T06:04:02.568401+0000","last_peered":"2024-03-08T06:04:02.568401+0000","last_clean":"2024-03-08T06:04:02.568401+0000","last_became_active":"2024-03-08T06:04:01.540657+0000","last_became_peered":"2024-03-08T06:04:01.540657+0000","last_unstale":"2024-03-08T06:04:02.568401+0000","last_undegraded":"2024-03-08T06:04:02.568401+0000","last_fullsized":"2024-03-08T06:04:02.568401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617381+0000","last_change":"2024-03-08T06:04:02.565226+0000","last_active":"2024-03-08T06:04:03.617381+0000","last_peered":"2024-03-08T06:04:03.617381+0000","last_clean":"2024-03-08T06:04:03.617381+0000","last_became_active":"2024-03-08T06:04:02.564692+0000","last_became_peered":"2024-03-08T06:04:02.564692+0000","last_unstale":"2024-03-08T06:04:03.617381+0000","last_undegraded":"2024-03-08T06:04:03.617381+0000","last_fullsized":"2024-03-08T06:04:03.617381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118978+0000","last_change":"2024-03-08T06:04:01.540655+0000","last_active":"2024-03-08T06:04:04.118978+0000","last_peered":"2024-03-08T06:04:04.118978+0000","last_clean":"2024-03-08T06:04:04.118978+0000","last_became_active":"2024-03-08T06:04:01.540205+0000","last_became_peered":"2024-03-08T06:04:01.540205+0000","last_unstale":"2024-03-08T06:04:04.118978+0000","last_undegraded":"2024-03-08T06:04:04.118978+0000","last_fullsized":"2024-03-08T06:04:04.118978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:12:32.122620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618419+0000","last_change":"2024-03-08T06:04:02.559001+0000","last_active":"2024-03-08T06:04:03.618419+0000","last_peered":"2024-03-08T06:04:03.618419+0000","last_clean":"2024-03-08T06:04:03.618419+0000","last_became_active":"2024-03-08T06:04:02.558681+0000","last_became_peered":"2024-03-08T06:04:02.558681+0000","last_unstale":"2024-03-08T06:04:03.618419+0000","last_undegraded":"2024-03-08T06:04:03.618419+0000","last_fullsized":"2024-03-08T06:04:03.618419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:19:21.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.570018+0000","last_change":"2024-03-08T06:04:01.542669+0000","last_active":"2024-03-08T06:04:02.570018+0000","last_peered":"2024-03-08T06:04:02.570018+0000","last_clean":"2024-03-08T06:04:02.570018+0000","last_became_active":"2024-03-08T06:04:01.540916+0000","last_became_peered":"2024-03-08T06:04:01.540916+0000","last_unstale":"2024-03-08T06:04:02.570018+0000","last_undegraded":"2024-03-08T06:04:02.570018+0000","last_fullsized":"2024-03-08T06:04:02.570018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618960+0000","last_change":"2024-03-08T06:04:02.569733+0000","last_active":"2024-03-08T06:04:03.618960+0000","last_peered":"2024-03-08T06:04:03.618960+0000","last_clean":"2024-03-08T06:04:03.618960+0000","last_became_active":"2024-03-08T06:04:02.567299+0000","last_became_peered":"2024-03-08T06:04:02.567299+0000","last_unstale":"2024-03-08T06:04:03.618960+0000","last_undegraded":"2024-03-08T06:04:03.618960+0000","last_fullsized":"2024-03-08T06:04:03.618960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:16:15.668237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553839+0000","last_change":"2024-03-08T06:04:01.544281+0000","last_active":"2024-03-08T06:04:03.553839+0000","last_peered":"2024-03-08T06:04:03.553839+0000","last_clean":"2024-03-08T06:04:03.553839+0000","last_became_active":"2024-03-08T06:04:01.543982+0000","last_became_peered":"2024-03-08T06:04:01.543982+0000","last_unstale":"2024-03-08T06:04:03.553839+0000","last_undegraded":"2024-03-08T06:04:03.553839+0000","last_fullsized":"2024-03-08T06:04:03.553839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:50.471516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516800+0000","last_change":"2024-03-08T06:04:02.551888+0000","last_active":"2024-03-08T06:04:03.516800+0000","last_peered":"2024-03-08T06:04:03.516800+0000","last_clean":"2024-03-08T06:04:03.516800+0000","last_became_active":"2024-03-08T06:04:02.551321+0000","last_became_peered":"2024-03-08T06:04:02.551321+0000","last_unstale":"2024-03-08T06:04:03.516800+0000","last_undegraded":"2024-03-08T06:04:03.516800+0000","last_fullsized":"2024-03-08T06:04:03.516800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:33:11.525956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.512617+0000","last_change":"2024-03-08T06:04:01.554656+0000","last_active":"2024-03-08T06:04:02.512617+0000","last_peered":"2024-03-08T06:04:02.512617+0000","last_clean":"2024-03-08T06:04:02.512617+0000","last_became_active":"2024-03-08T06:04:01.554326+0000","last_became_peered":"2024-03-08T06:04:01.554326+0000","last_unstale":"2024-03-08T06:04:02.512617+0000","last_undegraded":"2024-03-08T06:04:02.512617+0000","last_fullsized":"2024-03-08T06:04:02.512617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618096+0000","last_change":"2024-03-08T06:04:02.568164+0000","last_active":"2024-03-08T06:04:03.618096+0000","last_peered":"2024-03-08T06:04:03.618096+0000","last_clean":"2024-03-08T06:04:03.618096+0000","last_became_active":"2024-03-08T06:04:02.567123+0000","last_became_peered":"2024-03-08T06:04:02.567123+0000","last_unstale":"2024-03-08T06:04:03.618096+0000","last_undegraded":"2024-03-08T06:04:03.618096+0000","last_fullsized":"2024-03-08T06:04:03.618096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:42:12.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617235+0000","last_change":"2024-03-08T06:03:57.473597+0000","last_active":"2024-03-08T06:04:03.617235+0000","last_peered":"2024-03-08T06:04:03.617235+0000","last_clean":"2024-03-08T06:04:03.617235+0000","last_became_active":"2024-03-08T06:03:57.468593+0000","last_became_peered":"2024-03-08T06:03:57.468593+0000","last_unstale":"2024-03-08T06:04:03.617235+0000","last_undegraded":"2024-03-08T06:04:03.617235+0000","last_fullsized":"2024-03-08T06:04:03.617235+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_clean_scrub_stamp":"2024-03-08T06:03:56.214441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:45:28.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515695+0000","last_change":"2024-03-08T06:04:01.554640+0000","last_active":"2024-03-08T06:04:03.515695+0000","last_peered":"2024-03-08T06:04:03.515695+0000","last_clean":"2024-03-08T06:04:03.515695+0000","last_became_active":"2024-03-08T06:04:01.554274+0000","last_became_peered":"2024-03-08T06:04:01.554274+0000","last_unstale":"2024-03-08T06:04:03.515695+0000","last_undegraded":"2024-03-08T06:04:03.515695+0000","last_fullsized":"2024-03-08T06:04:03.515695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511891+0000","last_change":"2024-03-08T06:04:02.556830+0000","last_active":"2024-03-08T06:04:03.511891+0000","last_peered":"2024-03-08T06:04:03.511891+0000","last_clean":"2024-03-08T06:04:03.511891+0000","last_became_active":"2024-03-08T06:04:02.556071+0000","last_became_peered":"2024-03-08T06:04:02.556071+0000","last_unstale":"2024-03-08T06:04:03.511891+0000","last_undegraded":"2024-03-08T06:04:03.511891+0000","last_fullsized":"2024-03-08T06:04:03.511891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617015+0000","last_change":"2024-03-08T06:04:01.556875+0000","last_active":"2024-03-08T06:04:03.617015+0000","last_peered":"2024-03-08T06:04:03.617015+0000","last_clean":"2024-03-08T06:04:03.617015+0000","last_became_active":"2024-03-08T06:04:01.555346+0000","last_became_peered":"2024-03-08T06:04:01.555346+0000","last_unstale":"2024-03-08T06:04:03.617015+0000","last_undegraded":"2024-03-08T06:04:03.617015+0000","last_fullsized":"2024-03-08T06:04:03.617015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.566425+0000","last_change":"2024-03-08T06:04:02.563977+0000","last_active":"2024-03-08T06:04:02.566425+0000","last_peered":"2024-03-08T06:04:02.566425+0000","last_clean":"2024-03-08T06:04:02.566425+0000","last_became_active":"2024-03-08T06:04:02.563653+0000","last_became_peered":"2024-03-08T06:04:02.563653+0000","last_unstale":"2024-03-08T06:04:02.566425+0000","last_undegraded":"2024-03-08T06:04:02.566425+0000","last_fullsized":"2024-03-08T06:04:02.566425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:23:12.541838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.514447+0000","last_change":"2024-03-08T06:04:01.542592+0000","last_active":"2024-03-08T06:04:02.514447+0000","last_peered":"2024-03-08T06:04:02.514447+0000","last_clean":"2024-03-08T06:04:02.514447+0000","last_became_active":"2024-03-08T06:04:01.541567+0000","last_became_peered":"2024-03-08T06:04:01.541567+0000","last_unstale":"2024-03-08T06:04:02.514447+0000","last_undegraded":"2024-03-08T06:04:02.514447+0000","last_fullsized":"2024-03-08T06:04:02.514447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553855+0000","last_change":"2024-03-08T06:04:02.525242+0000","last_active":"2024-03-08T06:04:03.553855+0000","last_peered":"2024-03-08T06:04:03.553855+0000","last_clean":"2024-03-08T06:04:03.553855+0000","last_became_active":"2024-03-08T06:04:02.524598+0000","last_became_peered":"2024-03-08T06:04:02.524598+0000","last_unstale":"2024-03-08T06:04:03.553855+0000","last_undegraded":"2024-03-08T06:04:03.553855+0000","last_fullsized":"2024-03-08T06:04:03.553855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516807+0000","last_change":"2024-03-08T06:04:01.557363+0000","last_active":"2024-03-08T06:04:03.516807+0000","last_peered":"2024-03-08T06:04:03.516807+0000","last_clean":"2024-03-08T06:04:03.516807+0000","last_became_active":"2024-03-08T06:04:01.556516+0000","last_became_peered":"2024-03-08T06:04:01.556516+0000","last_unstale":"2024-03-08T06:04:03.516807+0000","last_undegraded":"2024-03-08T06:04:03.516807+0000","last_fullsized":"2024-03-08T06:04:03.516807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118240+0000","last_change":"2024-03-08T06:04:02.554738+0000","last_active":"2024-03-08T06:04:04.118240+0000","last_peered":"2024-03-08T06:04:04.118240+0000","last_clean":"2024-03-08T06:04:04.118240+0000","last_became_active":"2024-03-08T06:04:02.554446+0000","last_became_peered":"2024-03-08T06:04:02.554446+0000","last_unstale":"2024-03-08T06:04:04.118240+0000","last_undegraded":"2024-03-08T06:04:04.118240+0000","last_fullsized":"2024-03-08T06:04:04.118240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:51.718653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553561+0000","last_change":"2024-03-08T06:04:01.543134+0000","last_active":"2024-03-08T06:04:03.553561+0000","last_peered":"2024-03-08T06:04:03.553561+0000","last_clean":"2024-03-08T06:04:03.553561+0000","last_became_active":"2024-03-08T06:04:01.542768+0000","last_became_peered":"2024-03-08T06:04:01.542768+0000","last_unstale":"2024-03-08T06:04:03.553561+0000","last_undegraded":"2024-03-08T06:04:03.553561+0000","last_fullsized":"2024-03-08T06:04:03.553561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517531+0000","last_change":"2024-03-08T06:04:02.549139+0000","last_active":"2024-03-08T06:04:03.517531+0000","last_peered":"2024-03-08T06:04:03.517531+0000","last_clean":"2024-03-08T06:04:03.517531+0000","last_became_active":"2024-03-08T06:04:02.548765+0000","last_became_peered":"2024-03-08T06:04:02.548765+0000","last_unstale":"2024-03-08T06:04:03.517531+0000","last_undegraded":"2024-03-08T06:04:03.517531+0000","last_fullsized":"2024-03-08T06:04:03.517531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:41:43.397781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618664+0000","last_change":"2024-03-08T06:04:01.553438+0000","last_active":"2024-03-08T06:04:03.618664+0000","last_peered":"2024-03-08T06:04:03.618664+0000","last_clean":"2024-03-08T06:04:03.618664+0000","last_became_active":"2024-03-08T06:04:01.553120+0000","last_became_peered":"2024-03-08T06:04:01.553120+0000","last_unstale":"2024-03-08T06:04:03.618664+0000","last_undegraded":"2024-03-08T06:04:03.618664+0000","last_fullsized":"2024-03-08T06:04:03.618664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:05:58.801355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553556+0000","last_change":"2024-03-08T06:04:02.523565+0000","last_active":"2024-03-08T06:04:03.553556+0000","last_peered":"2024-03-08T06:04:03.553556+0000","last_clean":"2024-03-08T06:04:03.553556+0000","last_became_active":"2024-03-08T06:04:02.522603+0000","last_became_peered":"2024-03-08T06:04:02.522603+0000","last_unstale":"2024-03-08T06:04:03.553556+0000","last_undegraded":"2024-03-08T06:04:03.553556+0000","last_fullsized":"2024-03-08T06:04:03.553556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.557358+0000","last_change":"2024-03-08T06:04:01.559721+0000","last_active":"2024-03-08T06:04:03.557358+0000","last_peered":"2024-03-08T06:04:03.557358+0000","last_clean":"2024-03-08T06:04:03.557358+0000","last_became_active":"2024-03-08T06:04:01.559454+0000","last_became_peered":"2024-03-08T06:04:01.559454+0000","last_unstale":"2024-03-08T06:04:03.557358+0000","last_undegraded":"2024-03-08T06:04:03.557358+0000","last_fullsized":"2024-03-08T06:04:03.557358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512336+0000","last_change":"2024-03-08T06:04:02.555474+0000","last_active":"2024-03-08T06:04:03.512336+0000","last_peered":"2024-03-08T06:04:03.512336+0000","last_clean":"2024-03-08T06:04:03.512336+0000","last_became_active":"2024-03-08T06:04:02.551802+0000","last_became_peered":"2024-03-08T06:04:02.551802+0000","last_unstale":"2024-03-08T06:04:03.512336+0000","last_undegraded":"2024-03-08T06:04:03.512336+0000","last_fullsized":"2024-03-08T06:04:03.512336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559860+0000","last_change":"2024-03-08T06:04:01.559130+0000","last_active":"2024-03-08T06:04:03.559860+0000","last_peered":"2024-03-08T06:04:03.559860+0000","last_clean":"2024-03-08T06:04:03.559860+0000","last_became_active":"2024-03-08T06:04:01.558748+0000","last_became_peered":"2024-03-08T06:04:01.558748+0000","last_unstale":"2024-03-08T06:04:03.559860+0000","last_undegraded":"2024-03-08T06:04:03.559860+0000","last_fullsized":"2024-03-08T06:04:03.559860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.550035+0000","last_change":"2024-03-08T06:04:02.544456+0000","last_active":"2024-03-08T06:04:02.550035+0000","last_peered":"2024-03-08T06:04:02.550035+0000","last_clean":"2024-03-08T06:04:02.550035+0000","last_became_active":"2024-03-08T06:04:02.543887+0000","last_became_peered":"2024-03-08T06:04:02.543887+0000","last_unstale":"2024-03-08T06:04:02.550035+0000","last_undegraded":"2024-03-08T06:04:02.550035+0000","last_fullsized":"2024-03-08T06:04:02.550035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:15.104439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617019+0000","last_change":"2024-03-08T06:04:01.552997+0000","last_active":"2024-03-08T06:04:03.617019+0000","last_peered":"2024-03-08T06:04:03.617019+0000","last_clean":"2024-03-08T06:04:03.617019+0000","last_became_active":"2024-03-08T06:04:01.552617+0000","last_became_peered":"2024-03-08T06:04:01.552617+0000","last_unstale":"2024-03-08T06:04:03.617019+0000","last_undegraded":"2024-03-08T06:04:03.617019+0000","last_fullsized":"2024-03-08T06:04:03.617019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118081+0000","last_change":"2024-03-08T06:04:02.554162+0000","last_active":"2024-03-08T06:04:04.118081+0000","last_peered":"2024-03-08T06:04:04.118081+0000","last_clean":"2024-03-08T06:04:04.118081+0000","last_became_active":"2024-03-08T06:04:02.553850+0000","last_became_peered":"2024-03-08T06:04:02.553850+0000","last_unstale":"2024-03-08T06:04:04.118081+0000","last_undegraded":"2024-03-08T06:04:04.118081+0000","last_fullsized":"2024-03-08T06:04:04.118081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:09.964322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553380+0000","last_change":"2024-03-08T06:04:01.547872+0000","last_active":"2024-03-08T06:04:03.553380+0000","last_peered":"2024-03-08T06:04:03.553380+0000","last_clean":"2024-03-08T06:04:03.553380+0000","last_became_active":"2024-03-08T06:04:01.547553+0000","last_became_peered":"2024-03-08T06:04:01.547553+0000","last_unstale":"2024-03-08T06:04:03.553380+0000","last_undegraded":"2024-03-08T06:04:03.553380+0000","last_fullsized":"2024-03-08T06:04:03.553380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511556+0000","last_change":"2024-03-08T06:04:02.546841+0000","last_active":"2024-03-08T06:04:03.511556+0000","last_peered":"2024-03-08T06:04:03.511556+0000","last_clean":"2024-03-08T06:04:03.511556+0000","last_became_active":"2024-03-08T06:04:02.543164+0000","last_became_peered":"2024-03-08T06:04:02.543164+0000","last_unstale":"2024-03-08T06:04:03.511556+0000","last_undegraded":"2024-03-08T06:04:03.511556+0000","last_fullsized":"2024-03-08T06:04:03.511556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553503+0000","last_change":"2024-03-08T06:04:01.543186+0000","last_active":"2024-03-08T06:04:03.553503+0000","last_peered":"2024-03-08T06:04:03.553503+0000","last_clean":"2024-03-08T06:04:03.553503+0000","last_became_active":"2024-03-08T06:04:01.542185+0000","last_became_peered":"2024-03-08T06:04:01.542185+0000","last_unstale":"2024-03-08T06:04:03.553503+0000","last_undegraded":"2024-03-08T06:04:03.553503+0000","last_fullsized":"2024-03-08T06:04:03.553503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511983+0000","last_change":"2024-03-08T06:04:02.552865+0000","last_active":"2024-03-08T06:04:03.511983+0000","last_peered":"2024-03-08T06:04:03.511983+0000","last_clean":"2024-03-08T06:04:03.511983+0000","last_became_active":"2024-03-08T06:04:02.552563+0000","last_became_peered":"2024-03-08T06:04:02.552563+0000","last_unstale":"2024-03-08T06:04:03.511983+0000","last_undegraded":"2024-03-08T06:04:03.511983+0000","last_fullsized":"2024-03-08T06:04:03.511983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618134+0000","last_change":"2024-03-08T06:04:01.560452+0000","last_active":"2024-03-08T06:04:03.618134+0000","last_peered":"2024-03-08T06:04:03.618134+0000","last_clean":"2024-03-08T06:04:03.618134+0000","last_became_active":"2024-03-08T06:04:01.559883+0000","last_became_peered":"2024-03-08T06:04:01.559883+0000","last_unstale":"2024-03-08T06:04:03.618134+0000","last_undegraded":"2024-03-08T06:04:03.618134+0000","last_fullsized":"2024-03-08T06:04:03.618134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:50.471516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559002+0000","last_change":"2024-03-08T06:04:02.562767+0000","last_active":"2024-03-08T06:04:03.559002+0000","last_peered":"2024-03-08T06:04:03.559002+0000","last_clean":"2024-03-08T06:04:03.559002+0000","last_became_active":"2024-03-08T06:04:02.562425+0000","last_became_peered":"2024-03-08T06:04:02.562425+0000","last_unstale":"2024-03-08T06:04:03.559002+0000","last_undegraded":"2024-03-08T06:04:03.559002+0000","last_fullsized":"2024-03-08T06:04:03.559002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517395+0000","last_change":"2024-03-08T06:04:02.559515+0000","last_active":"2024-03-08T06:04:03.517395+0000","last_peered":"2024-03-08T06:04:03.517395+0000","last_clean":"2024-03-08T06:04:03.517395+0000","last_became_active":"2024-03-08T06:04:02.559082+0000","last_became_peered":"2024-03-08T06:04:02.559082+0000","last_unstale":"2024-03-08T06:04:03.517395+0000","last_undegraded":"2024-03-08T06:04:03.517395+0000","last_fullsized":"2024-03-08T06:04:03.517395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:42:12.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617836+0000","last_change":"2024-03-08T06:04:01.560587+0000","last_active":"2024-03-08T06:04:03.617836+0000","last_peered":"2024-03-08T06:04:03.617836+0000","last_clean":"2024-03-08T06:04:03.617836+0000","last_became_active":"2024-03-08T06:04:01.560275+0000","last_became_peered":"2024-03-08T06:04:01.560275+0000","last_unstale":"2024-03-08T06:04:03.617836+0000","last_undegraded":"2024-03-08T06:04:03.617836+0000","last_fullsized":"2024-03-08T06:04:03.617836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516385+0000","last_change":"2024-03-08T06:04:02.559339+0000","last_active":"2024-03-08T06:04:03.516385+0000","last_peered":"2024-03-08T06:04:03.516385+0000","last_clean":"2024-03-08T06:04:03.516385+0000","last_became_active":"2024-03-08T06:04:02.559023+0000","last_became_peered":"2024-03-08T06:04:02.559023+0000","last_unstale":"2024-03-08T06:04:03.516385+0000","last_undegraded":"2024-03-08T06:04:03.516385+0000","last_fullsized":"2024-03-08T06:04:03.516385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618371+0000","last_change":"2024-03-08T06:04:01.560256+0000","last_active":"2024-03-08T06:04:03.618371+0000","last_peered":"2024-03-08T06:04:03.618371+0000","last_clean":"2024-03-08T06:04:03.618371+0000","last_became_active":"2024-03-08T06:04:01.559917+0000","last_became_peered":"2024-03-08T06:04:01.559917+0000","last_unstale":"2024-03-08T06:04:03.618371+0000","last_undegraded":"2024-03-08T06:04:03.618371+0000","last_fullsized":"2024-03-08T06:04:03.618371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:55.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.564684+0000","last_change":"2024-03-08T06:04:02.551405+0000","last_active":"2024-03-08T06:04:02.564684+0000","last_peered":"2024-03-08T06:04:02.564684+0000","last_clean":"2024-03-08T06:04:02.564684+0000","last_became_active":"2024-03-08T06:04:02.550283+0000","last_became_peered":"2024-03-08T06:04:02.550283+0000","last_unstale":"2024-03-08T06:04:02.564684+0000","last_undegraded":"2024-03-08T06:04:02.564684+0000","last_fullsized":"2024-03-08T06:04:02.564684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:09.964322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512142+0000","last_change":"2024-03-08T06:04:01.540725+0000","last_active":"2024-03-08T06:04:03.512142+0000","last_peered":"2024-03-08T06:04:03.512142+0000","last_clean":"2024-03-08T06:04:03.512142+0000","last_became_active":"2024-03-08T06:04:01.540349+0000","last_became_peered":"2024-03-08T06:04:01.540349+0000","last_unstale":"2024-03-08T06:04:03.512142+0000","last_undegraded":"2024-03-08T06:04:03.512142+0000","last_fullsized":"2024-03-08T06:04:03.512142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512201+0000","last_change":"2024-03-08T06:04:02.554282+0000","last_active":"2024-03-08T06:04:03.512201+0000","last_peered":"2024-03-08T06:04:03.512201+0000","last_clean":"2024-03-08T06:04:03.512201+0000","last_became_active":"2024-03-08T06:04:02.551101+0000","last_became_peered":"2024-03-08T06:04:02.551101+0000","last_unstale":"2024-03-08T06:04:03.512201+0000","last_undegraded":"2024-03-08T06:04:03.512201+0000","last_fullsized":"2024-03-08T06:04:03.512201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T06:04:02.573670+0000","last_change":"2024-03-08T06:04:01.555646+0000","last_active":"2024-03-08T06:04:02.573670+0000","last_peered":"2024-03-08T06:04:02.573670+0000","last_clean":"2024-03-08T06:04:02.573670+0000","last_became_active":"2024-03-08T06:04:01.555404+0000","last_became_peered":"2024-03-08T06:04:01.555404+0000","last_unstale":"2024-03-08T06:04:02.573670+0000","last_undegraded":"2024-03-08T06:04:02.573670+0000","last_fullsized":"2024-03-08T06:04:02.573670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:52:36.740849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":69431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":69305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":811008,"data_stored":660311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:04:07.506 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-08T06:04:07.507 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-08T06:04:07.507 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-08T06:04:07.507 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-08T06:04:07.507 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-08T06:04:07.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.769+0000 7f9f54a00640 1 Processor -- start 2024-03-08T06:04:07.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.769+0000 7f9f54a00640 1 -- start start 2024-03-08T06:04:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f54a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9f5015b6c0 0x7f9f50161740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f54a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9f500b49e0 0x7f9f500adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f54a00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f50161d20 con 0x7f9f500ae400 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f54a00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f500b4e40 con 0x7f9f5015b6c0 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f54a00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5015e280 con 0x7f9f500b49e0 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f4f400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9f5015b6c0 0x7f9f50161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f4f400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9f5015b6c0 0x7f9f50161740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45550/0 (socket says 172.21.15.40:45550) 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f4f400640 1 -- 172.21.15.40:0/4092355496 learned_addr learned my addr 172.21.15.40:0/4092355496 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f4fe00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9f500ae400 0x7f9f500716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60812/0 (socket says 172.21.15.40:60812) 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f4f400640 1 -- 172.21.15.40:0/4092355496 >> v2:172.21.15.53:3300/0 conn(0x7f9f500b49e0 msgr2=0x7f9f500adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f4ea00640 1 --2- 172.21.15.40:0/4092355496 >> v2:172.21.15.53:3300/0 conn(0x7f9f500b49e0 0x7f9f500adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f4f400640 1 --2- 172.21.15.40:0/4092355496 >> v2:172.21.15.53:3300/0 conn(0x7f9f500b49e0 0x7f9f500adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f4f400640 1 -- 172.21.15.40:0/4092355496 >> v1:172.21.15.40:6789/0 conn(0x7f9f500ae400 legacy=0x7f9f500716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f4f400640 1 -- 172.21.15.40:0/4092355496 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50162000 con 0x7f9f5015b6c0 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f4ea00640 1 --2- 172.21.15.40:0/4092355496 >> v2:172.21.15.53:3300/0 conn(0x7f9f500b49e0 0x7f9f500adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:07.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.771+0000 7f9f4e000640 1 -- 172.21.15.40:0/4092355496 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303497216 0 0) 0x7f9f50161d20 con 0x7f9f500ae400 2024-03-08T06:04:07.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.773+0000 7f9f4f400640 1 --2- 172.21.15.40:0/4092355496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f5015b6c0 0x7f9f50161740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f5006f6a0 tx=0x7f9f4002b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6590aada661ee server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:07.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.773+0000 7f9f4e000640 1 -- 172.21.15.40:0/4092355496 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40034070 con 0x7f9f5015b6c0 2024-03-08T06:04:07.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.773+0000 7f9f4e000640 1 -- 172.21.15.40:0/4092355496 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f40030880 con 0x7f9f5015b6c0 2024-03-08T06:04:07.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.773+0000 7f9f4e000640 1 -- 172.21.15.40:0/4092355496 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40030c50 con 0x7f9f5015b6c0 2024-03-08T06:04:07.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.773+0000 7f9f54a00640 1 -- 172.21.15.40:0/4092355496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f5015b6c0 msgr2=0x7f9f50161740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:07.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.773+0000 7f9f54a00640 1 --2- 172.21.15.40:0/4092355496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f5015b6c0 0x7f9f50161740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f5006f6a0 tx=0x7f9f4002b9c0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.774+0000 7f9f54a00640 1 -- 172.21.15.40:0/4092355496 shutdown_connections 2024-03-08T06:04:07.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.774+0000 7f9f54a00640 1 --2- 172.21.15.40:0/4092355496 >> v2:172.21.15.53:3300/0 conn(0x7f9f500b49e0 0x7f9f500adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.774+0000 7f9f54a00640 1 --2- 172.21.15.40:0/4092355496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f5015b6c0 0x7f9f50161740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.774+0000 7f9f54a00640 1 -- 172.21.15.40:0/4092355496 >> 172.21.15.40:0/4092355496 conn(0x7f9f500b1850 msgr2=0x7f9f500b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:07.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.774+0000 7f9f54a00640 1 -- 172.21.15.40:0/4092355496 shutdown_connections 2024-03-08T06:04:07.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.774+0000 7f9f54a00640 1 -- 172.21.15.40:0/4092355496 wait complete. 2024-03-08T06:04:07.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.775+0000 7f9f54a00640 1 Processor -- start 2024-03-08T06:04:07.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.776+0000 7f9f54a00640 1 -- start start 2024-03-08T06:04:07.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f54a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f500ae400 0x7f9f50150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f54a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f500b49e0 0x7f9f500ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f54a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f5015b6c0 0x7f9f500ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f54a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f501625c0 con 0x7f9f500b49e0 2024-03-08T06:04:07.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f54a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500b4740 con 0x7f9f500ae400 2024-03-08T06:04:07.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f54a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5015baa0 con 0x7f9f5015b6c0 2024-03-08T06:04:07.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f4ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f500b49e0 0x7f9f500ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f4ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f500b49e0 0x7f9f500ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45554/0 (socket says 172.21.15.40:45554) 2024-03-08T06:04:07.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f4ea00640 1 -- 172.21.15.40:0/3774449869 learned_addr learned my addr 172.21.15.40:0/3774449869 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:07.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f4ea00640 1 -- 172.21.15.40:0/3774449869 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f5015b6c0 msgr2=0x7f9f500ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:07.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f4ea00640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f5015b6c0 0x7f9f500ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f4ea00640 1 -- 172.21.15.40:0/3774449869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f500ae400 msgr2=0x7f9f50150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:07.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f4ea00640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f500ae400 0x7f9f50150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:07.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.777+0000 7f9f4ea00640 1 -- 172.21.15.40:0/3774449869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5015a720 con 0x7f9f500b49e0 2024-03-08T06:04:07.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.778+0000 7f9f4ea00640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f500b49e0 0x7f9f500ceb90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00c820 tx=0x7f9f3c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:07.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.778+0000 7f9f47400640 1 -- 172.21.15.40:0/3774449869 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c015070 con 0x7f9f500b49e0 2024-03-08T06:04:07.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.778+0000 7f9f47400640 1 -- 172.21.15.40:0/3774449869 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c007a30 con 0x7f9f500b49e0 2024-03-08T06:04:07.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.778+0000 7f9f47400640 1 -- 172.21.15.40:0/3774449869 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c007e00 con 0x7f9f500b49e0 2024-03-08T06:04:07.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.778+0000 7f9f54a00640 1 -- 172.21.15.40:0/3774449869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f500ccf70 con 0x7f9f500b49e0 2024-03-08T06:04:07.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.778+0000 7f9f54a00640 1 -- 172.21.15.40:0/3774449869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f500cd530 con 0x7f9f500b49e0 2024-03-08T06:04:07.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.779+0000 7f9f54a00640 1 -- 172.21.15.40:0/3774449869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f50075100 con 0x7f9f500b49e0 2024-03-08T06:04:07.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.780+0000 7f9f47400640 1 -- 172.21.15.40:0/3774449869 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f3c018070 con 0x7f9f500b49e0 2024-03-08T06:04:07.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.781+0000 7f9f47400640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f3006b4d0 0x7f9f3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:07.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.781+0000 7f9f47400640 1 -- 172.21.15.40:0/3774449869 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f3c08a1e0 con 0x7f9f500b49e0 2024-03-08T06:04:07.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.781+0000 7f9f4f400640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f3006b4d0 0x7f9f3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:07.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.782+0000 7f9f4f400640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f3006b4d0 0x7f9f3006d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f40036000 tx=0x7f9f40030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:07.786+0000 7f9f47400640 1 -- 172.21.15.40:0/3774449869 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f5015a720 con 0x7f9f500b49e0 2024-03-08T06:04:08.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.033+0000 7f9f54a00640 1 -- 172.21.15.40:0/3774449869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9f500b1e40 con 0x7f9f500b49e0 2024-03-08T06:04:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.040+0000 7f9f47400640 1 -- 172.21.15.40:0/3774449869 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171782 (secure 0 0 0) 0x7f9f3c058fc0 con 0x7f9f500b49e0 2024-03-08T06:04:08.042 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.044+0000 7f9f54a00640 1 -- 172.21.15.40:0/3774449869 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f3006b4d0 msgr2=0x7f9f3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.044+0000 7f9f54a00640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f3006b4d0 0x7f9f3006d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f40036000 tx=0x7f9f40030000 comp rx=0 tx=0).stop 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.045+0000 7f9f54a00640 1 -- 172.21.15.40:0/3774449869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f500b49e0 msgr2=0x7f9f500ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.045+0000 7f9f54a00640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f500b49e0 0x7f9f500ceb90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00c820 tx=0x7f9f3c00cc20 comp rx=0 tx=0).stop 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.045+0000 7f9f54a00640 1 -- 172.21.15.40:0/3774449869 shutdown_connections 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.045+0000 7f9f54a00640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f3006b4d0 0x7f9f3006d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.045+0000 7f9f54a00640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f5015b6c0 0x7f9f500ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.045+0000 7f9f54a00640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f500b49e0 0x7f9f500ceb90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.045+0000 7f9f54a00640 1 --2- 172.21.15.40:0/3774449869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f500ae400 0x7f9f50150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.045+0000 7f9f54a00640 1 -- 172.21.15.40:0/3774449869 >> 172.21.15.40:0/3774449869 conn(0x7f9f500b1850 msgr2=0x7f9f5006dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.045+0000 7f9f54a00640 1 -- 172.21.15.40:0/3774449869 shutdown_connections 2024-03-08T06:04:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.045+0000 7f9f54a00640 1 -- 172.21.15.40:0/3774449869 wait complete. 2024-03-08T06:04:08.065 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":448564530},{"type":"v1","addr":"172.21.15.40:6833","nonce":448564530}]},"active_addr":"172.21.15.40:6833/448564530","active_change":"2024-03-08T06:03:47.464727+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":1650095000}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":3090258314}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":2202994500}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":1977554210}]}]} 2024-03-08T06:04:08.068 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-08T06:04:08.068 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-08T06:04:08.068 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:08.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.284+0000 7fa697200640 1 Processor -- start 2024-03-08T06:04:08.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.284+0000 7fa697200640 1 -- start start 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.286+0000 7fa697200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6981445d0 0x7fa69806deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.286+0000 7fa697200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa698071b80 0x7fa6980cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.286+0000 7fa697200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69806e3f0 con 0x7fa698144d50 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.286+0000 7fa697200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69806ebf0 con 0x7fa6981445d0 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.286+0000 7fa697200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69806eed0 con 0x7fa698071b80 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.286+0000 7fa695e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6981445d0 0x7fa69806deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.286+0000 7fa695e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6981445d0 0x7fa69806deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45558/0 (socket says 172.21.15.40:45558) 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.286+0000 7fa695e00640 1 -- 172.21.15.40:0/1388404232 learned_addr learned my addr 172.21.15.40:0/1388404232 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.287+0000 7fa695e00640 1 -- 172.21.15.40:0/1388404232 >> v2:172.21.15.53:3300/0 conn(0x7fa698071b80 msgr2=0x7fa6980cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.287+0000 7fa696800640 1 --2- 172.21.15.40:0/1388404232 >> v2:172.21.15.53:3300/0 conn(0x7fa698071b80 0x7fa6980cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.287+0000 7fa695e00640 1 --2- 172.21.15.40:0/1388404232 >> v2:172.21.15.53:3300/0 conn(0x7fa698071b80 0x7fa6980cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.287+0000 7fa695e00640 1 -- 172.21.15.40:0/1388404232 >> v1:172.21.15.53:6789/0 conn(0x7fa698144d50 legacy=0x7fa698071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:08.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.287+0000 7fa695e00640 1 -- 172.21.15.40:0/1388404232 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69806f1b0 con 0x7fa6981445d0 2024-03-08T06:04:08.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.287+0000 7fa696800640 1 --2- 172.21.15.40:0/1388404232 >> v2:172.21.15.53:3300/0 conn(0x7fa698071b80 0x7fa6980cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:08.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.288+0000 7fa695e00640 1 --2- 172.21.15.40:0/1388404232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6981445d0 0x7fa69806deb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa69806e6d0 tx=0x7fa68802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e0ace7896ec95d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:08.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.288+0000 7fa694a00640 1 -- 172.21.15.40:0/1388404232 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688035020 con 0x7fa6981445d0 2024-03-08T06:04:08.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.288+0000 7fa694a00640 1 -- 172.21.15.40:0/1388404232 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa688031870 con 0x7fa6981445d0 2024-03-08T06:04:08.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.288+0000 7fa694a00640 1 -- 172.21.15.40:0/1388404232 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688031c40 con 0x7fa6981445d0 2024-03-08T06:04:08.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.289+0000 7fa697200640 1 -- 172.21.15.40:0/1388404232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6981445d0 msgr2=0x7fa69806deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.289+0000 7fa697200640 1 --2- 172.21.15.40:0/1388404232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6981445d0 0x7fa69806deb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa69806e6d0 tx=0x7fa68802b910 comp rx=0 tx=0).stop 2024-03-08T06:04:08.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.289+0000 7fa697200640 1 -- 172.21.15.40:0/1388404232 shutdown_connections 2024-03-08T06:04:08.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.289+0000 7fa697200640 1 --2- 172.21.15.40:0/1388404232 >> v2:172.21.15.53:3300/0 conn(0x7fa698071b80 0x7fa6980cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.289+0000 7fa697200640 1 --2- 172.21.15.40:0/1388404232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6981445d0 0x7fa69806deb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.289+0000 7fa697200640 1 -- 172.21.15.40:0/1388404232 >> 172.21.15.40:0/1388404232 conn(0x7fa6980b1600 msgr2=0x7fa6980b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.289+0000 7fa697200640 1 -- 172.21.15.40:0/1388404232 shutdown_connections 2024-03-08T06:04:08.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.289+0000 7fa697200640 1 -- 172.21.15.40:0/1388404232 wait complete. 2024-03-08T06:04:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.291+0000 7fa697200640 1 Processor -- start 2024-03-08T06:04:08.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.291+0000 7fa697200640 1 -- start start 2024-03-08T06:04:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.292+0000 7fa697200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa698071b80 0x7fa69815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.292+0000 7fa697200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6981445d0 0x7fa6980b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.292+0000 7fa697200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa698144d50 0x7fa6980cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.292+0000 7fa697200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6981449b0 con 0x7fa698144d50 2024-03-08T06:04:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.292+0000 7fa697200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6980cc340 con 0x7fa6981445d0 2024-03-08T06:04:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.292+0000 7fa697200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa698144b30 con 0x7fa698071b80 2024-03-08T06:04:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.292+0000 7fa696800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa698144d50 0x7fa6980cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.292+0000 7fa696800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa698144d50 0x7fa6980cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45568/0 (socket says 172.21.15.40:45568) 2024-03-08T06:04:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.292+0000 7fa696800640 1 -- 172.21.15.40:0/1124585078 learned_addr learned my addr 172.21.15.40:0/1124585078 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa695e00640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa698071b80 0x7fa69815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa695400640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6981445d0 0x7fa6980b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa696800640 1 -- 172.21.15.40:0/1124585078 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa698071b80 msgr2=0x7fa69815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa696800640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa698071b80 0x7fa69815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa696800640 1 -- 172.21.15.40:0/1124585078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6981445d0 msgr2=0x7fa6980b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa696800640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6981445d0 0x7fa6980b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa696800640 1 -- 172.21.15.40:0/1124585078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6981e5690 con 0x7fa698144d50 2024-03-08T06:04:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa695400640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6981445d0 0x7fa6980b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa696800640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa698144d50 0x7fa6980cc860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa69000cd50 tx=0x7fa69000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa68e000640 1 -- 172.21.15.40:0/1124585078 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000f680 con 0x7fa698144d50 2024-03-08T06:04:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa68e000640 1 -- 172.21.15.40:0/1124585078 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69000f840 con 0x7fa698144d50 2024-03-08T06:04:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa68e000640 1 -- 172.21.15.40:0/1124585078 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000fc30 con 0x7fa698144d50 2024-03-08T06:04:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.293+0000 7fa697200640 1 -- 172.21.15.40:0/1124585078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6980b49d0 con 0x7fa698144d50 2024-03-08T06:04:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.294+0000 7fa697200640 1 -- 172.21.15.40:0/1124585078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6980b2e90 con 0x7fa698144d50 2024-03-08T06:04:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.295+0000 7fa697200640 1 -- 172.21.15.40:0/1124585078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa698075100 con 0x7fa698144d50 2024-03-08T06:04:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.296+0000 7fa68e000640 1 -- 172.21.15.40:0/1124585078 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6900213f0 con 0x7fa698144d50 2024-03-08T06:04:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.297+0000 7fa68e000640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa67806b520 0x7fa67806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.297+0000 7fa68e000640 1 -- 172.21.15.40:0/1124585078 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa69008b080 con 0x7fa698144d50 2024-03-08T06:04:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.297+0000 7fa695e00640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa67806b520 0x7fa67806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.298+0000 7fa695e00640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa67806b520 0x7fa67806d9e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa688036000 tx=0x7fa68803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:08.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.301+0000 7fa68e000640 1 -- 172.21.15.40:0/1124585078 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa690014030 con 0x7fa698144d50 2024-03-08T06:04:08.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.499+0000 7fa697200640 1 -- 172.21.15.40:0/1124585078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6980af350 con 0x7fa698144d50 2024-03-08T06:04:08.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.502+0000 7fa68e000640 1 -- 172.21.15.40:0/1124585078 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7fa690059e50 con 0x7fa698144d50 2024-03-08T06:04:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:08.503 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.505+0000 7fa697200640 1 -- 172.21.15.40:0/1124585078 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa67806b520 msgr2=0x7fa67806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.505+0000 7fa697200640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa67806b520 0x7fa67806d9e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa688036000 tx=0x7fa68803c040 comp rx=0 tx=0).stop 2024-03-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.505+0000 7fa697200640 1 -- 172.21.15.40:0/1124585078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa698144d50 msgr2=0x7fa6980cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.505+0000 7fa697200640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa698144d50 0x7fa6980cc860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa69000cd50 tx=0x7fa69000a650 comp rx=0 tx=0).stop 2024-03-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.505+0000 7fa697200640 1 -- 172.21.15.40:0/1124585078 shutdown_connections 2024-03-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.505+0000 7fa697200640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa67806b520 0x7fa67806d9e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.505+0000 7fa697200640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa698144d50 0x7fa6980cc860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.505+0000 7fa697200640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6981445d0 0x7fa6980b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.505+0000 7fa697200640 1 --2- 172.21.15.40:0/1124585078 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa698071b80 0x7fa69815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.505+0000 7fa697200640 1 -- 172.21.15.40:0/1124585078 >> 172.21.15.40:0/1124585078 conn(0x7fa6980b1600 msgr2=0x7fa698071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.505+0000 7fa697200640 1 -- 172.21.15.40:0/1124585078 shutdown_connections 2024-03-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.506+0000 7fa697200640 1 -- 172.21.15.40:0/1124585078 wait complete. 2024-03-08T06:04:08.526 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-08T06:04:08.526 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.742+0000 7f4c9de00640 1 Processor -- start 2024-03-08T06:04:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.742+0000 7f4c9de00640 1 -- start start 2024-03-08T06:04:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.744+0000 7f4c9de00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4c98144d50 0x7f4c98071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.744+0000 7f4c9de00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c98071d90 con 0x7f4c980719b0 2024-03-08T06:04:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.744+0000 7f4c9de00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9806ebf0 con 0x7f4c981445d0 2024-03-08T06:04:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.744+0000 7f4c9de00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9806eed0 con 0x7f4c98144d50 2024-03-08T06:04:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.744+0000 7f4c9d400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4c980719b0 0x7f4c980cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60820/0 (socket says 172.21.15.40:60820) 2024-03-08T06:04:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.744+0000 7f4c9d400640 1 -- 172.21.15.40:0/3939806905 learned_addr learned my addr 172.21.15.40:0/3939806905 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.744+0000 7f4c97e00640 1 --2- 172.21.15.40:0/3939806905 >> v2:172.21.15.53:3300/0 conn(0x7f4c98144d50 0x7f4c98071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.744+0000 7f4c97400640 1 -- 172.21.15.40:0/3939806905 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164108814 0 0) 0x7f4c98071d90 con 0x7f4c980719b0 2024-03-08T06:04:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.744+0000 7f4c97400640 1 -- 172.21.15.40:0/3939806905 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c80003520 con 0x7f4c980719b0 2024-03-08T06:04:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.745+0000 7f4c97e00640 1 -- 172.21.15.40:0/3939806905 >> v1:172.21.15.53:6789/0 conn(0x7f4c981445d0 legacy=0x7f4c9806deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.745+0000 7f4c97e00640 1 -- 172.21.15.40:0/3939806905 >> v1:172.21.15.40:6789/0 conn(0x7f4c980719b0 legacy=0x7f4c980cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.745+0000 7f4c97400640 1 -- 172.21.15.40:0/3939806905 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 906307260 0 0) 0x7f4c80003520 con 0x7f4c980719b0 2024-03-08T06:04:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.745+0000 7f4c97e00640 1 -- 172.21.15.40:0/3939806905 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9806f1b0 con 0x7f4c98144d50 2024-03-08T06:04:08.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.746+0000 7f4c97e00640 1 --2- 172.21.15.40:0/3939806905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c98144d50 0x7f4c98071470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c840097a0 tx=0x7f4c8402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26e0c1c4c0203b7a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:08.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.747+0000 7f4c97400640 1 -- 172.21.15.40:0/3939806905 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84038070 con 0x7f4c98144d50 2024-03-08T06:04:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.747+0000 7f4c97400640 1 -- 172.21.15.40:0/3939806905 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c84031500 con 0x7f4c98144d50 2024-03-08T06:04:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.747+0000 7f4c97400640 1 -- 172.21.15.40:0/3939806905 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c900019f0 con 0x7f4c98144d50 2024-03-08T06:04:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.747+0000 7f4c9de00640 1 -- 172.21.15.40:0/3939806905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c98144d50 msgr2=0x7f4c98071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.747+0000 7f4c9de00640 1 --2- 172.21.15.40:0/3939806905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c98144d50 0x7f4c98071470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c840097a0 tx=0x7f4c8402b820 comp rx=0 tx=0).stop 2024-03-08T06:04:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.748+0000 7f4c9de00640 1 -- 172.21.15.40:0/3939806905 shutdown_connections 2024-03-08T06:04:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.748+0000 7f4c9de00640 1 --2- 172.21.15.40:0/3939806905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c98144d50 0x7f4c98071470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.748+0000 7f4c9de00640 1 -- 172.21.15.40:0/3939806905 >> 172.21.15.40:0/3939806905 conn(0x7f4c980b1600 msgr2=0x7f4c980b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.748+0000 7f4c9de00640 1 -- 172.21.15.40:0/3939806905 shutdown_connections 2024-03-08T06:04:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.748+0000 7f4c9de00640 1 -- 172.21.15.40:0/3939806905 wait complete. 2024-03-08T06:04:08.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.749+0000 7f4c9de00640 1 Processor -- start 2024-03-08T06:04:08.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.750+0000 7f4c9de00640 1 -- start start 2024-03-08T06:04:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.750+0000 7f4c9de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c980719b0 0x7f4c9813c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.750+0000 7f4c9de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c981445d0 0x7f4c980b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.750+0000 7f4c9de00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c98144d50 0x7f4c9815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.750+0000 7f4c9de00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c981449b0 con 0x7f4c980719b0 2024-03-08T06:04:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.750+0000 7f4c9de00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980cc340 con 0x7f4c981445d0 2024-03-08T06:04:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.750+0000 7f4c9de00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c98144b30 con 0x7f4c98144d50 2024-03-08T06:04:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.751+0000 7f4c9ca00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c980719b0 0x7f4c9813c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.751+0000 7f4c9ca00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c980719b0 0x7f4c9813c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45584/0 (socket says 172.21.15.40:45584) 2024-03-08T06:04:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.751+0000 7f4c9ca00640 1 -- 172.21.15.40:0/1083753038 learned_addr learned my addr 172.21.15.40:0/1083753038 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.751+0000 7f4c97e00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c981445d0 0x7f4c980b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.751+0000 7f4c9ca00640 1 -- 172.21.15.40:0/1083753038 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c98144d50 msgr2=0x7f4c9815c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.751+0000 7f4c9ca00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c98144d50 0x7f4c9815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.751+0000 7f4c9ca00640 1 -- 172.21.15.40:0/1083753038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c981445d0 msgr2=0x7f4c980b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.751+0000 7f4c9ca00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c981445d0 0x7f4c980b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.751+0000 7f4c9ca00640 1 -- 172.21.15.40:0/1083753038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c980b2a10 con 0x7f4c980719b0 2024-03-08T06:04:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.751+0000 7f4c9ca00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c980719b0 0x7f4c9813c270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00c400 tx=0x7f4c8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.752+0000 7f4c94c00640 1 -- 172.21.15.40:0/1083753038 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c017070 con 0x7f4c980719b0 2024-03-08T06:04:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.752+0000 7f4c94c00640 1 -- 172.21.15.40:0/1083753038 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c0052d0 con 0x7f4c980719b0 2024-03-08T06:04:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.752+0000 7f4c94c00640 1 -- 172.21.15.40:0/1083753038 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c003830 con 0x7f4c980719b0 2024-03-08T06:04:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.752+0000 7f4c9de00640 1 -- 172.21.15.40:0/1083753038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c980b2cf0 con 0x7f4c980719b0 2024-03-08T06:04:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.752+0000 7f4c9de00640 1 -- 172.21.15.40:0/1083753038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c980b32b0 con 0x7f4c980719b0 2024-03-08T06:04:08.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.753+0000 7f4c9de00640 1 -- 172.21.15.40:0/1083753038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c98071450 con 0x7f4c980719b0 2024-03-08T06:04:08.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.754+0000 7f4c94c00640 1 -- 172.21.15.40:0/1083753038 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c8c01a070 con 0x7f4c980719b0 2024-03-08T06:04:08.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.755+0000 7f4c94c00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4c7806b4d0 0x7f4c7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:08.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.755+0000 7f4c94c00640 1 -- 172.21.15.40:0/1083753038 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c8c089570 con 0x7f4c980719b0 2024-03-08T06:04:08.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.756+0000 7f4c97e00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4c7806b4d0 0x7f4c7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:08.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.756+0000 7f4c97e00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4c7806b4d0 0x7f4c7806d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c84035000 tx=0x7f4c84008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:08.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.760+0000 7f4c94c00640 1 -- 172.21.15.40:0/1083753038 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c8c0583e0 con 0x7f4c980719b0 2024-03-08T06:04:08.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.959+0000 7f4c9de00640 1 -- 172.21.15.40:0/1083753038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c980af350 con 0x7f4c980719b0 2024-03-08T06:04:08.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.962+0000 7f4c94c00640 1 -- 172.21.15.40:0/1083753038 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f4c8c087e20 con 0x7f4c980719b0 2024-03-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:08.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 -- 172.21.15.40:0/1083753038 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4c7806b4d0 msgr2=0x7f4c7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4c7806b4d0 0x7f4c7806d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c84035000 tx=0x7f4c84008030 comp rx=0 tx=0).stop 2024-03-08T06:04:08.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 -- 172.21.15.40:0/1083753038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c980719b0 msgr2=0x7f4c9813c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:08.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c980719b0 0x7f4c9813c270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00c400 tx=0x7f4c8c00c8c0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 -- 172.21.15.40:0/1083753038 shutdown_connections 2024-03-08T06:04:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4c7806b4d0 0x7f4c7806d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c98144d50 0x7f4c9815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c981445d0 0x7f4c980b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 --2- 172.21.15.40:0/1083753038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c980719b0 0x7f4c9813c270 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 -- 172.21.15.40:0/1083753038 >> 172.21.15.40:0/1083753038 conn(0x7f4c980b1600 msgr2=0x7f4c980b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 -- 172.21.15.40:0/1083753038 shutdown_connections 2024-03-08T06:04:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:08.965+0000 7f4c9de00640 1 -- 172.21.15.40:0/1083753038 wait complete. 2024-03-08T06:04:08.985 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-08T06:04:08.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-08T06:04:08.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-08T06:04:08.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-08T06:04:08.987 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-08T06:04:08.987 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-08T06:04:08.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-08T06:04:08.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-08T06:04:09.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.283+0000 7f80ae400640 1 Processor -- start 2024-03-08T06:04:09.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.284+0000 7f80ae400640 1 -- start start 2024-03-08T06:04:09.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.286+0000 7f80ae400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80a81611f0 0x7f80a80b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.286+0000 7f80ae400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80a80d0ff0 0x7f80a80b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.286+0000 7f80ae400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a80d3410 con 0x7f80a80b5a20 2024-03-08T06:04:09.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.286+0000 7f80ae400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a80ceac0 con 0x7f80a80d0ff0 2024-03-08T06:04:09.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.286+0000 7f80ae400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80cf080 con 0x7f80a81611f0 2024-03-08T06:04:09.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.287+0000 7f80a7e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80a80d0ff0 0x7f80a80b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.287+0000 7f80a7e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80a80d0ff0 0x7f80a80b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:49522/0 (socket says 172.21.15.40:49522) 2024-03-08T06:04:09.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.287+0000 7f80a7e00640 1 -- 172.21.15.40:0/1181183170 learned_addr learned my addr 172.21.15.40:0/1181183170 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.287+0000 7f80ad000640 1 --2- 172.21.15.40:0/1181183170 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80a81611f0 0x7f80a80b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.288+0000 7f80ad000640 1 -- 172.21.15.40:0/1181183170 >> v2:172.21.15.53:3300/0 conn(0x7f80a80d0ff0 msgr2=0x7f80a80b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.288+0000 7f80ad000640 1 --2- 172.21.15.40:0/1181183170 >> v2:172.21.15.53:3300/0 conn(0x7f80a80d0ff0 0x7f80a80b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.288+0000 7f80ad000640 1 -- 172.21.15.40:0/1181183170 >> v1:172.21.15.40:6789/0 conn(0x7f80a80b5a20 legacy=0x7f80a81619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.288+0000 7f80ad000640 1 -- 172.21.15.40:0/1181183170 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a80d36f0 con 0x7f80a81611f0 2024-03-08T06:04:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.288+0000 7f80a7e00640 1 --2- 172.21.15.40:0/1181183170 >> v2:172.21.15.53:3300/0 conn(0x7f80a80d0ff0 0x7f80a80b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.291+0000 7f80ad000640 1 --2- 172.21.15.40:0/1181183170 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80a81611f0 0x7f80a80b27a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8098009d90 tx=0x7f809802bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e971be4a2f44a0fb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.292+0000 7f80a7400640 1 -- 172.21.15.40:0/1181183170 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098039070 con 0x7f80a81611f0 2024-03-08T06:04:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.292+0000 7f80a7400640 1 -- 172.21.15.40:0/1181183170 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809802cb50 con 0x7f80a81611f0 2024-03-08T06:04:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.292+0000 7f80a7400640 1 -- 172.21.15.40:0/1181183170 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098034440 con 0x7f80a81611f0 2024-03-08T06:04:09.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.293+0000 7f80ae400640 1 -- 172.21.15.40:0/1181183170 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80a81611f0 msgr2=0x7f80a80b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.293+0000 7f80ae400640 1 --2- 172.21.15.40:0/1181183170 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80a81611f0 0x7f80a80b27a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8098009d90 tx=0x7f809802bda0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.293+0000 7f80ae400640 1 -- 172.21.15.40:0/1181183170 shutdown_connections 2024-03-08T06:04:09.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.293+0000 7f80ae400640 1 --2- 172.21.15.40:0/1181183170 >> v2:172.21.15.53:3300/0 conn(0x7f80a80d0ff0 0x7f80a80b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.293+0000 7f80ae400640 1 --2- 172.21.15.40:0/1181183170 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80a81611f0 0x7f80a80b27a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.293+0000 7f80ae400640 1 -- 172.21.15.40:0/1181183170 >> 172.21.15.40:0/1181183170 conn(0x7f80a806c700 msgr2=0x7f80a806cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.295+0000 7f3990400640 1 Processor -- start 2024-03-08T06:04:09.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.295+0000 7f47ade00640 1 Processor -- start 2024-03-08T06:04:09.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.296+0000 7f47ade00640 1 -- start start 2024-03-08T06:04:09.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.296+0000 7f2b9e800640 1 Processor -- start 2024-03-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.296+0000 7f80ae400640 1 -- 172.21.15.40:0/1181183170 shutdown_connections 2024-03-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.296+0000 7f2b9e800640 1 -- start start 2024-03-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.296+0000 7f80ae400640 1 -- 172.21.15.40:0/1181183170 wait complete. 2024-03-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.296+0000 7f768d400640 1 Processor -- start 2024-03-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.297+0000 7f768d400640 1 -- start start 2024-03-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.295+0000 7f3990400640 1 -- start start 2024-03-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.297+0000 7f3990400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f398806de80 0x7f39880718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.297+0000 7f3990400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398806d3e0 con 0x7f3988154390 2024-03-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.297+0000 7f3990400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39880cc760 con 0x7f3988071df0 2024-03-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.297+0000 7f3990400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3988072260 con 0x7f398806de80 2024-03-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.298+0000 7f47ade00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47a80718b0 0x7f47a8071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.299+0000 7f80ae400640 1 Processor -- start 2024-03-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.298+0000 7f47ade00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a80cc760 con 0x7f47a806b770 2024-03-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.298+0000 7f47ade00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a806d680 con 0x7f47a806de80 2024-03-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.298+0000 7f47ade00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a806d9f0 con 0x7f47a80718b0 2024-03-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.298+0000 7f47a6a00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f47a806b770 0x7f47a815adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60846/0 (socket says 172.21.15.40:60846) 2024-03-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.298+0000 7f47a6a00640 1 -- 172.21.15.40:0/2384165142 learned_addr learned my addr 172.21.15.40:0/2384165142 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.299+0000 7f398d000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f398806de80 0x7f39880718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.299+0000 7f2b9e800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2b98142a40 0x7f2b9815d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.299+0000 7f47a7400640 1 --2- 172.21.15.40:0/2384165142 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47a80718b0 0x7f47a8071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.299+0000 7f398e400640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3988071df0 0x7f39880b3020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.40:38698/0 (socket says 172.21.15.40:38698) 2024-03-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.299+0000 7f398e400640 1 -- 172.21.15.40:0/1468073427 learned_addr learned my addr 172.21.15.40:0/1468073427 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.299+0000 7f2b9e800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b980b3f30 0x7f2b9815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.299+0000 7f80ae400640 1 -- start start 2024-03-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.299+0000 7f2b9e800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b980b3490 con 0x7f2b98142ef0 2024-03-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f2b97400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2b98142a40 0x7f2b9815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f768d400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7688071f90 0x7f768815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f2b97400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2b98142a40 0x7f2b9815d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:51154/0 (socket says 172.21.15.40:51154) 2024-03-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f2b97400640 1 -- 172.21.15.40:0/3597795170 learned_addr learned my addr 172.21.15.40:0/3597795170 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f2b9e800640 1 -- 172.21.15.40:0/3597795170 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9815e450 con 0x7f2b980b3f30 2024-03-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126079272 0 0) 0x7f39880cc760 con 0x7f3988071df0 2024-03-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f768d400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76880ccb90 con 0x7f76880b4c60 2024-03-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f47a7400640 1 -- 172.21.15.40:0/2384165142 >> v1:172.21.15.53:6789/0 conn(0x7f47a806de80 legacy=0x7f47a8154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f47a7400640 1 -- 172.21.15.40:0/2384165142 >> v1:172.21.15.40:6789/0 conn(0x7f47a806b770 legacy=0x7f47a815adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f47a7400640 1 -- 172.21.15.40:0/2384165142 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80cc430 con 0x7f47a80718b0 2024-03-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f2b9e800640 1 -- 172.21.15.40:0/3597795170 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b3730 con 0x7f2b98142a40 2024-03-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f768d400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76880cc5d0 con 0x7f7688157860 2024-03-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f768d400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768815de10 con 0x7f7688071f90 2024-03-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3970003520 con 0x7f3988071df0 2024-03-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f7686800640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7688157860 0x7f768815e2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.40:38710/0 (socket says 172.21.15.40:38710) 2024-03-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.300+0000 7f2b8ea00640 1 --2- 172.21.15.40:0/3597795170 >> v2:172.21.15.53:3300/0 conn(0x7f2b980b3f30 0x7f2b9815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f7686800640 1 -- 172.21.15.40:0/3250488240 learned_addr learned my addr 172.21.15.40:0/3250488240 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f7687200640 1 --2- 172.21.15.40:0/3250488240 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7688071f90 0x7f768815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 <== mon.1 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2818016986 0 0) 0x7f3970003520 con 0x7f3988071df0 2024-03-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f398806de80 msgr2=0x7f39880718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 --2- 172.21.15.40:0/1468073427 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f398806de80 0x7f39880718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 >> v1:172.21.15.40:6789/0 conn(0x7f3988154390 legacy=0x7f398815adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398806d680 con 0x7f3988071df0 2024-03-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 <== mon.1 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f39800019f0 con 0x7f3988071df0 2024-03-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 >> v1:172.21.15.53:6789/0 conn(0x7f3988071df0 legacy=0x7f39880b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 --2- 172.21.15.40:0/1468073427 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f398806de80 0x7f39700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 --2- 172.21.15.40:0/1468073427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970003f10 0x7f3970004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 --2- 172.21.15.40:0/1468073427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3970004d40 0x7f39700051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3988072260 con 0x7f3970003f10 2024-03-08T06:04:09.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397000bbb0 con 0x7f3970004d40 2024-03-08T06:04:09.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.301+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397000bdb0 con 0x7f398806de80 2024-03-08T06:04:09.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f2b97400640 1 -- 172.21.15.40:0/3597795170 >> v2:172.21.15.53:3300/0 conn(0x7f2b980b3f30 msgr2=0x7f2b9815ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f2b97400640 1 --2- 172.21.15.40:0/3597795170 >> v2:172.21.15.53:3300/0 conn(0x7f2b980b3f30 0x7f2b9815ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f2b97400640 1 -- 172.21.15.40:0/3597795170 >> v1:172.21.15.40:6789/0 conn(0x7f2b98142ef0 legacy=0x7f2b9806d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f80ae400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b5a20 0x7f80a80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f80ae400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a80d0ff0 0x7f80a815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f80ae400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80a81611f0 0x7f80a815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f80ae400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a81615d0 con 0x7f80a80b5a20 2024-03-08T06:04:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f80ae400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80d1450 con 0x7f80a80d0ff0 2024-03-08T06:04:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f80ae400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8161750 con 0x7f80a81611f0 2024-03-08T06:04:09.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f398d000640 1 --2- 172.21.15.40:0/1468073427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970003f10 0x7f3970004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f398da00640 1 --2- 172.21.15.40:0/1468073427 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f398806de80 0x7f39700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f2b97400640 1 -- 172.21.15.40:0/3597795170 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b980b3a10 con 0x7f2b98142a40 2024-03-08T06:04:09.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f80ad000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b5a20 0x7f80a80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f47a7400640 1 --2- 172.21.15.40:0/2384165142 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47a80718b0 0x7f47a8071ce0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f47940097d0 tx=0x7f479402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d0a3108fc0c23d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f80ad000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b5a20 0x7f80a80cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45592/0 (socket says 172.21.15.40:45592) 2024-03-08T06:04:09.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f80ad000640 1 -- 172.21.15.40:0/1967806962 learned_addr learned my addr 172.21.15.40:0/1967806962 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f7685400640 1 -- 172.21.15.40:0/3250488240 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389943154 0 0) 0x7f76880cc5d0 con 0x7f7688157860 2024-03-08T06:04:09.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.302+0000 7f7685400640 1 -- 172.21.15.40:0/3250488240 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f766c003500 con 0x7f7688157860 2024-03-08T06:04:09.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f398e400640 1 --2- 172.21.15.40:0/1468073427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3970004d40 0x7f39700051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f47a5600640 1 -- 172.21.15.40:0/2384165142 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794031780 con 0x7f47a80718b0 2024-03-08T06:04:09.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f47a5600640 1 -- 172.21.15.40:0/2384165142 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4794037070 con 0x7f47a80718b0 2024-03-08T06:04:09.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f47a5600640 1 -- 172.21.15.40:0/2384165142 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794031b40 con 0x7f47a80718b0 2024-03-08T06:04:09.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f398d000640 1 -- 172.21.15.40:0/1468073427 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f398806de80 msgr2=0x7f39700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f398d000640 1 --2- 172.21.15.40:0/1468073427 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f398806de80 0x7f39700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f398d000640 1 -- 172.21.15.40:0/1468073427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3970004d40 msgr2=0x7f39700051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f398d000640 1 --2- 172.21.15.40:0/1468073427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3970004d40 0x7f39700051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f398d000640 1 -- 172.21.15.40:0/1468073427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398806d680 con 0x7f3970003f10 2024-03-08T06:04:09.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f80a7e00640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a80d0ff0 0x7f80a815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f80ada00640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80a81611f0 0x7f80a815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.304+0000 7f80ad000640 1 -- 172.21.15.40:0/1967806962 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80a81611f0 msgr2=0x7f80a815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.304+0000 7f80ad000640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80a81611f0 0x7f80a815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.304+0000 7f80ad000640 1 -- 172.21.15.40:0/1967806962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a80d0ff0 msgr2=0x7f80a815ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.304+0000 7f80ad000640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80a80d0ff0 0x7f80a815ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.304+0000 7f80ad000640 1 -- 172.21.15.40:0/1967806962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a81581a0 con 0x7f80a80b5a20 2024-03-08T06:04:09.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f7687200640 1 -- 172.21.15.40:0/3250488240 >> v1:172.21.15.53:6789/0 conn(0x7f7688157860 legacy=0x7f768815e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f7687200640 1 -- 172.21.15.40:0/3250488240 >> v1:172.21.15.40:6789/0 conn(0x7f76880b4c60 legacy=0x7f7688071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.303+0000 7f7687200640 1 -- 172.21.15.40:0/3250488240 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76880cc8b0 con 0x7f7688071f90 2024-03-08T06:04:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.305+0000 7f80ad000640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b5a20 0x7f80a80cca70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8098009d90 tx=0x7f8098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.305+0000 7f7687200640 1 --2- 172.21.15.40:0/3250488240 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7688071f90 0x7f768815ad90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7678009e40 tx=0x7f767802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c78694660ada74a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.305+0000 7f398d000640 1 --2- 172.21.15.40:0/1468073427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970003f10 0x7f3970004800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3974009ac0 tx=0x7f3970005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9d9ea659bfeff server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f2b97400640 1 --2- 172.21.15.40:0/3597795170 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2b98142a40 0x7f2b9815d880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800c2e0 tx=0x7f2b8802e410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cda3f640587e4bf server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.304+0000 7f47ade00640 1 -- 172.21.15.40:0/2384165142 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47a80718b0 msgr2=0x7f47a8071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f7685400640 1 -- 172.21.15.40:0/3250488240 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678039070 con 0x7f7688071f90 2024-03-08T06:04:09.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f7685400640 1 -- 172.21.15.40:0/3250488240 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767802cc50 con 0x7f7688071f90 2024-03-08T06:04:09.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f7685400640 1 -- 172.21.15.40:0/3250488240 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678034660 con 0x7f7688071f90 2024-03-08T06:04:09.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.304+0000 7f47ade00640 1 --2- 172.21.15.40:0/2384165142 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47a80718b0 0x7f47a8071ce0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f47940097d0 tx=0x7f479402b820 comp rx=0 tx=0).stop 2024-03-08T06:04:09.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.305+0000 7f80a4c00640 1 -- 172.21.15.40:0/1967806962 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098044020 con 0x7f80a80b5a20 2024-03-08T06:04:09.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f80a4c00640 1 -- 172.21.15.40:0/1967806962 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809802cd00 con 0x7f80a80b5a20 2024-03-08T06:04:09.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f80a4c00640 1 -- 172.21.15.40:0/1967806962 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098039040 con 0x7f80a80b5a20 2024-03-08T06:04:09.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f80ae400640 1 -- 172.21.15.40:0/1967806962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a8158480 con 0x7f80a80b5a20 2024-03-08T06:04:09.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f80ae400640 1 -- 172.21.15.40:0/1967806962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a8158a40 con 0x7f80a80b5a20 2024-03-08T06:04:09.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f2b96a00640 1 -- 172.21.15.40:0/3597795170 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8803a070 con 0x7f2b98142a40 2024-03-08T06:04:09.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f2b96a00640 1 -- 172.21.15.40:0/3597795170 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8800a4e0 con 0x7f2b98142a40 2024-03-08T06:04:09.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f2b96a00640 1 -- 172.21.15.40:0/3597795170 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8800a8b0 con 0x7f2b98142a40 2024-03-08T06:04:09.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f768d400640 1 -- 172.21.15.40:0/3250488240 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7688071f90 msgr2=0x7f768815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f768d400640 1 --2- 172.21.15.40:0/3250488240 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7688071f90 0x7f768815ad90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7678009e40 tx=0x7f767802bea0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f768d400640 1 -- 172.21.15.40:0/3250488240 shutdown_connections 2024-03-08T06:04:09.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f768d400640 1 --2- 172.21.15.40:0/3250488240 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7688071f90 0x7f768815ad90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f768d400640 1 -- 172.21.15.40:0/3250488240 >> 172.21.15.40:0/3250488240 conn(0x7f768806ae30 msgr2=0x7f768806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.305+0000 7f47ade00640 1 -- 172.21.15.40:0/2384165142 shutdown_connections 2024-03-08T06:04:09.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.305+0000 7f47ade00640 1 --2- 172.21.15.40:0/2384165142 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47a80718b0 0x7f47a8071ce0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.305+0000 7f47ade00640 1 -- 172.21.15.40:0/2384165142 >> 172.21.15.40:0/2384165142 conn(0x7f47a806ac50 msgr2=0x7f47a806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.305+0000 7f47ade00640 1 -- 172.21.15.40:0/2384165142 shutdown_connections 2024-03-08T06:04:09.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.305+0000 7f47ade00640 1 -- 172.21.15.40:0/2384165142 wait complete. 2024-03-08T06:04:09.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f47ade00640 1 Processor -- start 2024-03-08T06:04:09.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974039070 con 0x7f3970003f10 2024-03-08T06:04:09.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397402c6b0 con 0x7f3970003f10 2024-03-08T06:04:09.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.306+0000 7f3987e00640 1 -- 172.21.15.40:0/1468073427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974039070 con 0x7f3970003f10 2024-03-08T06:04:09.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f3990400640 1 -- 172.21.15.40:0/1468073427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970003f10 msgr2=0x7f3970004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f3990400640 1 --2- 172.21.15.40:0/1468073427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970003f10 0x7f3970004800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3974009ac0 tx=0x7f3970005830 comp rx=0 tx=0).stop 2024-03-08T06:04:09.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f47ade00640 1 -- start start 2024-03-08T06:04:09.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.307+0000 7f398e400640 1 -- 172.21.15.40:0/1468073427 reap_dead start 2024-03-08T06:04:09.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.308+0000 7f3990400640 1 -- 172.21.15.40:0/1468073427 shutdown_connections 2024-03-08T06:04:09.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.308+0000 7f3990400640 1 -- 172.21.15.40:0/1468073427 >> 172.21.15.40:0/1468073427 conn(0x7f398806ac50 msgr2=0x7f398806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.308+0000 7f768d400640 1 -- 172.21.15.40:0/3250488240 shutdown_connections 2024-03-08T06:04:09.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.309+0000 7f3990400640 1 -- 172.21.15.40:0/1468073427 shutdown_connections 2024-03-08T06:04:09.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.309+0000 7f768d400640 1 -- 172.21.15.40:0/3250488240 wait complete. 2024-03-08T06:04:09.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.310+0000 7f47ade00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a806b770 0x7f47a8140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.309+0000 7f80ae400640 1 -- 172.21.15.40:0/1967806962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80a80cd560 con 0x7f80a80b5a20 2024-03-08T06:04:09.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.311+0000 7f80a4c00640 1 -- 172.21.15.40:0/1967806962 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8098047020 con 0x7f80a80b5a20 2024-03-08T06:04:09.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.310+0000 7f47ade00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47a806de80 0x7f47a8143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.310+0000 7f47ade00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80718b0 0x7f47a813fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.310+0000 7f47ade00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8072220 con 0x7f47a80718b0 2024-03-08T06:04:09.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.311+0000 7f47a6000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47a806de80 0x7f47a8143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.311+0000 7f47ade00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8154830 con 0x7f47a806b770 2024-03-08T06:04:09.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.310+0000 7f47a6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a806b770 0x7f47a8140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.311+0000 7f47a6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a806b770 0x7f47a8140660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:49562/0 (socket says 172.21.15.40:49562) 2024-03-08T06:04:09.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.311+0000 7f47a6a00640 1 -- 172.21.15.40:0/2094695507 learned_addr learned my addr 172.21.15.40:0/2094695507 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.311+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80b4bf0 con 0x7f47a806de80 2024-03-08T06:04:09.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.311+0000 7f47a7400640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80718b0 0x7f47a813fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.309+0000 7f3990400640 1 -- 172.21.15.40:0/1468073427 wait complete. 2024-03-08T06:04:09.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.308+0000 7f2b9e800640 1 -- 172.21.15.40:0/3597795170 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2b98142a40 msgr2=0x7f2b9815d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.308+0000 7f2b9e800640 1 --2- 172.21.15.40:0/3597795170 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2b98142a40 0x7f2b9815d880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800c2e0 tx=0x7f2b8802e410 comp rx=0 tx=0).stop 2024-03-08T06:04:09.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.309+0000 7f2b9e800640 1 -- 172.21.15.40:0/3597795170 shutdown_connections 2024-03-08T06:04:09.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.309+0000 7f2b9e800640 1 --2- 172.21.15.40:0/3597795170 >> v2:172.21.15.53:3300/0 conn(0x7f2b980b3f30 0x7f2b9815ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.309+0000 7f2b9e800640 1 --2- 172.21.15.40:0/3597795170 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2b98142a40 0x7f2b9815d880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.309+0000 7f2b9e800640 1 -- 172.21.15.40:0/3597795170 >> 172.21.15.40:0/3597795170 conn(0x7f2b980b16e0 msgr2=0x7f2b980b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.309+0000 7f2b9e800640 1 -- 172.21.15.40:0/3597795170 shutdown_connections 2024-03-08T06:04:09.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.309+0000 7f2b9e800640 1 -- 172.21.15.40:0/3597795170 wait complete. 2024-03-08T06:04:09.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.310+0000 7feef1c00640 1 Processor -- start 2024-03-08T06:04:09.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.310+0000 7feef1c00640 1 -- start start 2024-03-08T06:04:09.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7f2b9e800640 1 Processor -- start 2024-03-08T06:04:09.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7f47a6a00640 1 -- 172.21.15.40:0/2094695507 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47a806de80 msgr2=0x7f47a8143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7f47a6a00640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47a806de80 0x7f47a8143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7f768d400640 1 Processor -- start 2024-03-08T06:04:09.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7f47a6a00640 1 -- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80718b0 msgr2=0x7f47a813fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7f47a6a00640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80718b0 0x7f47a813fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7f47a6a00640 1 -- 172.21.15.40:0/2094695507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8140380 con 0x7f47a806b770 2024-03-08T06:04:09.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7f768d400640 1 -- start start 2024-03-08T06:04:09.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7f2b9e800640 1 -- start start 2024-03-08T06:04:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7f47a6a00640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a806b770 0x7f47a8140660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4798015d40 tx=0x7f4798013560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7feef1c00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feeec1631c0 0x7feeec071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.312+0000 7feef1c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feeec071940 0x7feeec0b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.313+0000 7feef1c00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec06d500 con 0x7feeec0aee90 2024-03-08T06:04:09.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.313+0000 7feef1c00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec06de30 con 0x7feeec071940 2024-03-08T06:04:09.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.313+0000 7feef1c00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec071d70 con 0x7feeec1631c0 2024-03-08T06:04:09.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.313+0000 7feef1200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feeec071940 0x7feeec0b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.313+0000 7feeebe00640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feeec0aee90 0x7feeec0b44c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.40:38714/0 (socket says 172.21.15.40:38714) 2024-03-08T06:04:09.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.313+0000 7feeebe00640 1 -- 172.21.15.40:0/1958397326 learned_addr learned my addr 172.21.15.40:0/1958397326 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:09.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7feeeb400640 1 --2- 172.21.15.40:0/1958397326 >> v2:172.21.15.53:3300/0 conn(0x7feeec1631c0 0x7feeec071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7f19f8a00640 1 Processor -- start 2024-03-08T06:04:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7f19f8a00640 1 -- start start 2024-03-08T06:04:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.313+0000 7f479f400640 1 -- 172.21.15.40:0/2094695507 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479801d020 con 0x7f47a806b770 2024-03-08T06:04:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.313+0000 7f479f400640 1 -- 172.21.15.40:0/2094695507 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4798019630 con 0x7f47a806b770 2024-03-08T06:04:09.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.313+0000 7f479f400640 1 -- 172.21.15.40:0/2094695507 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798012030 con 0x7f47a806b770 2024-03-08T06:04:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7feef1200640 1 -- 172.21.15.40:0/1958397326 >> v2:172.21.15.53:3300/0 conn(0x7feeec1631c0 msgr2=0x7feeec071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a8140e00 con 0x7f47a806b770 2024-03-08T06:04:09.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7feef1200640 1 --2- 172.21.15.40:0/1958397326 >> v2:172.21.15.53:3300/0 conn(0x7feeec1631c0 0x7feeec071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7f768d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7688071f90 0x7f76880ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7feef1200640 1 -- 172.21.15.40:0/1958397326 >> v1:172.21.15.53:6789/0 conn(0x7feeec0aee90 legacy=0x7feeec0b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7feef1200640 1 -- 172.21.15.40:0/1958397326 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec06d7e0 con 0x7feeec071940 2024-03-08T06:04:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7f3990400640 1 Processor -- start 2024-03-08T06:04:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7f80a4c00640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f808406b4d0 0x7f808406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f80a4c00640 1 -- 172.21.15.40:0/1967806962 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80980ac990 con 0x7f80a80b5a20 2024-03-08T06:04:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f80a4c00640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] conn(0x7f80840736f0 0x7f8084075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f80a4c00640 1 -- 172.21.15.40:0/1967806962 --> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80980020b0 con 0x7f80840736f0 2024-03-08T06:04:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f80a4c00640 1 -- 172.21.15.40:0/1967806962 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80980b12a0 con 0x7f80a80b5a20 2024-03-08T06:04:09.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.314+0000 7f768d400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880b4c60 0x7f76880a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f768d400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7688157860 0x7f76881ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f768d400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768806f6b0 con 0x7f76880b4c60 2024-03-08T06:04:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f768d400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688157c40 con 0x7f7688071f90 2024-03-08T06:04:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a81413c0 con 0x7f47a806b770 2024-03-08T06:04:09.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f3990400640 1 -- start start 2024-03-08T06:04:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f2b9e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b980b3f30 0x7f2b9813a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f2b9e800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2b98142a40 0x7f2b98134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f2b9e800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b98142ef0 0x7f2b981fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f2b9e800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b98072520 con 0x7f2b98142ef0 2024-03-08T06:04:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f2b9e800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b4c00 con 0x7f2b980b3f30 2024-03-08T06:04:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f2b9e800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980726a0 con 0x7f2b98142a40 2024-03-08T06:04:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f2b97400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b980b3f30 0x7f2b9813a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f2b97400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b980b3f30 0x7f2b9813a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:49586/0 (socket says 172.21.15.40:49586) 2024-03-08T06:04:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f2b97400640 1 -- 172.21.15.40:0/661265619 learned_addr learned my addr 172.21.15.40:0/661265619 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7ff1e8600640 1 Processor -- start 2024-03-08T06:04:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7ff1e8600640 1 -- start start 2024-03-08T06:04:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.316+0000 7f2b8ea00640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2b98142a40 0x7f2b98134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f768d400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768806f830 con 0x7f7688157860 2024-03-08T06:04:09.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f7686800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7688071f90 0x7f76880ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f7686800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7688071f90 0x7f76880ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:49580/0 (socket says 172.21.15.40:49580) 2024-03-08T06:04:09.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f7686800640 1 -- 172.21.15.40:0/3408334096 learned_addr learned my addr 172.21.15.40:0/3408334096 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.315+0000 7f7687200640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7688157860 0x7f76881ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.316+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47a8140ba0 con 0x7f47a806b770 2024-03-08T06:04:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.316+0000 7f80ada00640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] conn(0x7f80840736f0 0x7f8084075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.316+0000 7f7685e00640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880b4c60 0x7f76880a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.316+0000 7f7686800640 1 -- 172.21.15.40:0/3408334096 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7688157860 msgr2=0x7f76881ef6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.316+0000 7f7686800640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7688157860 0x7f76881ef6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f7686800640 1 -- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880b4c60 msgr2=0x7f76880a97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f7686800640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880b4c60 0x7f76880a97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f7686800640 1 -- 172.21.15.40:0/3408334096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76880ad600 con 0x7f7688071f90 2024-03-08T06:04:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.316+0000 7f2b97e00640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b98142ef0 0x7f2b981fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f19f8a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19f406de80 0x7f19f40718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f80a7e00640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f808406b4d0 0x7f808406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f2b97e00640 1 -- 172.21.15.40:0/661265619 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2b98142a40 msgr2=0x7f2b98134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7ff1e8600640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1e015df80 0x7ff1e00ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f7686800640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7688071f90 0x7f76880ad020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7680001b70 tx=0x7f7680014420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7ff1e8600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff1e00b6e30 0x7ff1e00d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7ff1e8600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e00d34a0 con 0x7ff1e0161ae0 2024-03-08T06:04:09.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7ff1e8600640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e00d2000 con 0x7ff1e015df80 2024-03-08T06:04:09.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f2b97e00640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2b98142a40 0x7f2b98134cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f2b97e00640 1 -- 172.21.15.40:0/661265619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b980b3f30 msgr2=0x7f2b9813a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f2b97e00640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b980b3f30 0x7f2b9813a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7ff1e8600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00ce9a0 con 0x7ff1e00b6e30 2024-03-08T06:04:09.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f3990400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970003f10 0x7f39881f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f2b97e00640 1 -- 172.21.15.40:0/661265619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9813a7e0 con 0x7f2b98142ef0 2024-03-08T06:04:09.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f19f8a00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f406d3e0 con 0x7f19f4154390 2024-03-08T06:04:09.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f19f8a00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f40cc760 con 0x7f19f4071df0 2024-03-08T06:04:09.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.317+0000 7f19f8a00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f406d9f0 con 0x7f19f406de80 2024-03-08T06:04:09.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f19f2800640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f19f4071df0 0x7f19f40b3020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.40:38718/0 (socket says 172.21.15.40:38718) 2024-03-08T06:04:09.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f19f2800640 1 -- 172.21.15.40:0/118632612 learned_addr learned my addr 172.21.15.40:0/118632612 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f3990400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39881413e0 0x7f39881ef690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f19f1400640 1 --2- 172.21.15.40:0/118632612 >> v2:172.21.15.53:3300/0 conn(0x7f19f406de80 0x7f19f40718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f80a7e00640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f808406b4d0 0x7f808406d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f80a815dfe0 tx=0x7f80a0009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f767ea00640 1 -- 172.21.15.40:0/3408334096 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680001d20 con 0x7f7688071f90 2024-03-08T06:04:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f767ea00640 1 -- 172.21.15.40:0/3408334096 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7680014d40 con 0x7f7688071f90 2024-03-08T06:04:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f767ea00640 1 -- 172.21.15.40:0/3408334096 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680005ab0 con 0x7f7688071f90 2024-03-08T06:04:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f3990400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3988141860 0x7f398813ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f3990400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39880cc430 con 0x7f3970003f10 2024-03-08T06:04:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f3990400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398806d9f0 con 0x7f39881413e0 2024-03-08T06:04:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f3990400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398806e330 con 0x7f3988141860 2024-03-08T06:04:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f398da00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970003f10 0x7f39881f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f398da00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970003f10 0x7f39881f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45638/0 (socket says 172.21.15.40:45638) 2024-03-08T06:04:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f398da00640 1 -- 172.21.15.40:0/29750587 learned_addr learned my addr 172.21.15.40:0/29750587 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f398d000640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39881413e0 0x7f39881ef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f2b97400640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b980b3f30 0x7f2b9813a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:09.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7feef1200640 1 --2- 172.21.15.40:0/1958397326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feeec071940 0x7feeec0b2890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feeec0722c0 tx=0x7feee402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42079a2efa4fc32 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f19f0a00640 1 -- 172.21.15.40:0/118632612 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151217297 0 0) 0x7f19f40cc760 con 0x7f19f4071df0 2024-03-08T06:04:09.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f19f0a00640 1 -- 172.21.15.40:0/118632612 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19d8003520 con 0x7f19f4071df0 2024-03-08T06:04:09.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f19f0a00640 1 -- 172.21.15.40:0/118632612 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347574524 0 0) 0x7f19f406d3e0 con 0x7f19f4154390 2024-03-08T06:04:09.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f398e400640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3988141860 0x7f398813ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7ff1e5c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1e015df80 0x7ff1e00ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f19f0a00640 1 -- 172.21.15.40:0/118632612 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19f40cc760 con 0x7f19f4154390 2024-03-08T06:04:09.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7ff1e5c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1e015df80 0x7ff1e00ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45628/0 (socket says 172.21.15.40:45628) 2024-03-08T06:04:09.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7ff1e5c00640 1 -- 172.21.15.40:0/3082295458 learned_addr learned my addr 172.21.15.40:0/3082295458 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f80ada00640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] conn(0x7f80840736f0 0x7f8084075b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.318+0000 7f768d400640 1 -- 172.21.15.40:0/3408334096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76880ad8e0 con 0x7f7688071f90 2024-03-08T06:04:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f768d400640 1 -- 172.21.15.40:0/3408334096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76880adea0 con 0x7f7688071f90 2024-03-08T06:04:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7feeeaa00640 1 -- 172.21.15.40:0/1958397326 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4039070 con 0x7feeec071940 2024-03-08T06:04:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.320+0000 7feeeaa00640 1 -- 172.21.15.40:0/1958397326 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee402cc70 con 0x7feeec071940 2024-03-08T06:04:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f398da00640 1 -- 172.21.15.40:0/29750587 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3988141860 msgr2=0x7f398813ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f398da00640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3988141860 0x7f398813ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f398da00640 1 -- 172.21.15.40:0/29750587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39881413e0 msgr2=0x7f39881ef690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f398da00640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39881413e0 0x7f39881ef690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f398da00640 1 -- 172.21.15.40:0/29750587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39881f0150 con 0x7f3970003f10 2024-03-08T06:04:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.320+0000 7ff1e5200640 1 --2- 172.21.15.40:0/3082295458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff1e00b6e30 0x7ff1e00d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.320+0000 7f80a4c00640 1 -- 172.21.15.40:0/1967806962 <== osd.0 v2:172.21.15.40:6800/671313654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f80980020b0 con 0x7f80840736f0 2024-03-08T06:04:09.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.320+0000 7f2b97e00640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b98142ef0 0x7f2b981fbc20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b410 tx=0x7f2b9000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7f768d400640 1 -- 172.21.15.40:0/3408334096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7688067cb0 con 0x7f7688071f90 2024-03-08T06:04:09.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7feef1c00640 1 -- 172.21.15.40:0/1958397326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feeec071940 msgr2=0x7feeec0b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7feef1c00640 1 --2- 172.21.15.40:0/1958397326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feeec071940 0x7feeec0b2890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feeec0722c0 tx=0x7feee402bec0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7f398da00640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970003f10 0x7f39881f36b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f397c00e4b0 tx=0x7f397c00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7ff1e5c00640 1 -- 172.21.15.40:0/3082295458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff1e00b6e30 msgr2=0x7ff1e00d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7feef1c00640 1 -- 172.21.15.40:0/1958397326 shutdown_connections 2024-03-08T06:04:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7feef1c00640 1 --2- 172.21.15.40:0/1958397326 >> v2:172.21.15.53:3300/0 conn(0x7feeec1631c0 0x7feeec071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7feef1c00640 1 --2- 172.21.15.40:0/1958397326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feeec071940 0x7feeec0b2890 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7feef1c00640 1 -- 172.21.15.40:0/1958397326 >> 172.21.15.40:0/1958397326 conn(0x7feeec0b1590 msgr2=0x7feeec0b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f19f1400640 1 -- 172.21.15.40:0/118632612 >> v1:172.21.15.53:6789/0 conn(0x7f19f4071df0 legacy=0x7f19f40b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f19f1400640 1 -- 172.21.15.40:0/118632612 >> v1:172.21.15.40:6789/0 conn(0x7f19f4154390 legacy=0x7f19f415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f19f1400640 1 -- 172.21.15.40:0/118632612 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f406d680 con 0x7f19f406de80 2024-03-08T06:04:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7ff1e5c00640 1 --2- 172.21.15.40:0/3082295458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff1e00b6e30 0x7ff1e00d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7ff1e5c00640 1 -- 172.21.15.40:0/3082295458 >> v1:172.21.15.40:6789/0 conn(0x7ff1e0161ae0 legacy=0x7ff1e00b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7ff1e5c00640 1 -- 172.21.15.40:0/3082295458 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00cec40 con 0x7ff1e015df80 2024-03-08T06:04:09.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.319+0000 7f479f400640 1 -- 172.21.15.40:0/2094695507 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4798020020 con 0x7f47a806b770 2024-03-08T06:04:09.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7f479f400640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f478806b4d0 0x7f478806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7f479f400640 1 -- 172.21.15.40:0/2094695507 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4798092f30 con 0x7f47a806b770 2024-03-08T06:04:09.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7feef1c00640 1 -- 172.21.15.40:0/1958397326 shutdown_connections 2024-03-08T06:04:09.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7feef1c00640 1 -- 172.21.15.40:0/1958397326 wait complete. 2024-03-08T06:04:09.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7f479f400640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] conn(0x7f47880736f0 0x7f4788075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7f479f400640 1 -- 172.21.15.40:0/2094695507 --> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47880761c0 con 0x7f47880736f0 2024-03-08T06:04:09.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7f479f400640 1 -- 172.21.15.40:0/2094695507 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4798093320 con 0x7f47a806b770 2024-03-08T06:04:09.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.322+0000 7f19f1400640 1 --2- 172.21.15.40:0/118632612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f406de80 0x7f19f40718b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00aea0 tx=0x7f19ec02ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b73b89c86edcd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7f3985600640 1 -- 172.21.15.40:0/29750587 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c0135f0 con 0x7f3970003f10 2024-03-08T06:04:09.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7f3985600640 1 -- 172.21.15.40:0/29750587 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c01a070 con 0x7f3970003f10 2024-03-08T06:04:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7f3985600640 1 -- 172.21.15.40:0/29750587 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c0139f0 con 0x7f3970003f10 2024-03-08T06:04:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.321+0000 7f3990400640 1 -- 172.21.15.40:0/29750587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39881f0430 con 0x7f3970003f10 2024-03-08T06:04:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.322+0000 7f47a7400640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] conn(0x7f47880736f0 0x7f4788075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.322+0000 7f47a6000640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f478806b4d0 0x7f478806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.322+0000 7f3990400640 1 -- 172.21.15.40:0/29750587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39881f3190 con 0x7f3970003f10 2024-03-08T06:04:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.322+0000 7f2b8fe00640 1 -- 172.21.15.40:0/661265619 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90015020 con 0x7f2b98142ef0 2024-03-08T06:04:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.322+0000 7f2b8fe00640 1 -- 172.21.15.40:0/661265619 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b90011850 con 0x7f2b98142ef0 2024-03-08T06:04:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.322+0000 7f2b8fe00640 1 -- 172.21.15.40:0/661265619 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90011c40 con 0x7f2b98142ef0 2024-03-08T06:04:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.323+0000 7f47a7400640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] conn(0x7f47880736f0 0x7f4788075b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.323+0000 7f19f0a00640 1 -- 172.21.15.40:0/118632612 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0004e60 con 0x7f19f406de80 2024-03-08T06:04:09.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.323+0000 7f19f0a00640 1 -- 172.21.15.40:0/118632612 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec02d790 con 0x7f19f406de80 2024-03-08T06:04:09.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.324+0000 7feef1c00640 1 Processor -- start 2024-03-08T06:04:09.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.323+0000 7f397aa00640 1 -- 172.21.15.40:0/29750587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f395c001030 con 0x7f3970003f10 2024-03-08T06:04:09.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.324+0000 7f2b9e800640 1 -- 172.21.15.40:0/661265619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b981356c0 con 0x7f2b98142ef0 2024-03-08T06:04:09.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.324+0000 7f2b9e800640 1 -- 172.21.15.40:0/661265619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b98135c80 con 0x7f2b98142ef0 2024-03-08T06:04:09.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.324+0000 7f19f0a00640 1 -- 172.21.15.40:0/118632612 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0004e60 con 0x7f19f406de80 2024-03-08T06:04:09.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.324+0000 7feef1c00640 1 -- start start 2024-03-08T06:04:09.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7f19f8a00640 1 -- 172.21.15.40:0/118632612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f406de80 msgr2=0x7f19f40718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7f19f8a00640 1 --2- 172.21.15.40:0/118632612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f406de80 0x7f19f40718b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00aea0 tx=0x7f19ec02ce70 comp rx=0 tx=0).stop 2024-03-08T06:04:09.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7f2b9e800640 1 -- 172.21.15.40:0/661265619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b98067cb0 con 0x7f2b98142ef0 2024-03-08T06:04:09.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7f19f8a00640 1 -- 172.21.15.40:0/118632612 shutdown_connections 2024-03-08T06:04:09.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7feef1c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeec071940 0x7feeec145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7f19f8a00640 1 --2- 172.21.15.40:0/118632612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f406de80 0x7f19f40718b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7f479f400640 1 -- 172.21.15.40:0/2094695507 <== osd.2 v2:172.21.15.40:6819/1467838663 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f47880761c0 con 0x7f47880736f0 2024-03-08T06:04:09.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7ff1e5c00640 1 --2- 172.21.15.40:0/3082295458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e015df80 0x7ff1e00ce460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00d2180 tx=0x7ff1d802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3224982d3b0c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7f19f8a00640 1 -- 172.21.15.40:0/118632612 >> 172.21.15.40:0/118632612 conn(0x7f19f406ac50 msgr2=0x7f19f406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7feef1c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feeec0aee90 0x7feeec1fb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7f19f8a00640 1 -- 172.21.15.40:0/118632612 shutdown_connections 2024-03-08T06:04:09.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7f19f8a00640 1 -- 172.21.15.40:0/118632612 wait complete. 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.325+0000 7feef1c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feeec1631c0 0x7feeec163670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.326+0000 7feef1c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0b4d10 con 0x7feeec0aee90 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.326+0000 7feeebe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeec071940 0x7feeec145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.326+0000 7feef1c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0af270 con 0x7feeec071940 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.326+0000 7feeebe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeec071940 0x7feeec145780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:49608/0 (socket says 172.21.15.40:49608) 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.326+0000 7feeebe00640 1 -- 172.21.15.40:0/2644208553 learned_addr learned my addr 172.21.15.40:0/2644208553 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.326+0000 7feef1c00640 1 -- 172.21.15.40:0/2644208553 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec06e2c0 con 0x7feeec1631c0 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.326+0000 7feef1200640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feeec1631c0 0x7feeec163670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.327+0000 7feeebe00640 1 -- 172.21.15.40:0/2644208553 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feeec1631c0 msgr2=0x7feeec163670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.327+0000 7feeebe00640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feeec1631c0 0x7feeec163670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.327+0000 7feeebe00640 1 -- 172.21.15.40:0/2644208553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feeec0aee90 msgr2=0x7feeec1fb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.327+0000 7feeebe00640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feeec0aee90 0x7feeec1fb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.327+0000 7feeebe00640 1 -- 172.21.15.40:0/2644208553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec1fc250 con 0x7feeec071940 2024-03-08T06:04:09.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.326+0000 7f767ea00640 1 -- 172.21.15.40:0/3408334096 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7680005c50 con 0x7f7688071f90 2024-03-08T06:04:09.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.327+0000 7f767ea00640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f766406b520 0x7f766406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.327+0000 7f767ea00640 1 -- 172.21.15.40:0/3408334096 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7680092620 con 0x7f7688071f90 2024-03-08T06:04:09.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.328+0000 7feeebe00640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeec071940 0x7feeec145780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feed800c400 tx=0x7feed800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.328+0000 7f19f8a00640 1 Processor -- start 2024-03-08T06:04:09.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.328+0000 7f767ea00640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] conn(0x7f7664073740 0x7f7664075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.328+0000 7f767ea00640 1 -- 172.21.15.40:0/3408334096 --> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7680021e50 con 0x7f7664073740 2024-03-08T06:04:09.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.328+0000 7f767ea00640 1 -- 172.21.15.40:0/3408334096 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7680092b00 con 0x7f7688071f90 2024-03-08T06:04:09.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.328+0000 7f7685e00640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f766406b520 0x7f766406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.328+0000 7f7687200640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] conn(0x7f7664073740 0x7f7664075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.329+0000 7f19f8a00640 1 -- start start 2024-03-08T06:04:09.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.329+0000 7feee3e00640 1 -- 172.21.15.40:0/2644208553 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed8017070 con 0x7feeec071940 2024-03-08T06:04:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.329+0000 7feee3e00640 1 -- 172.21.15.40:0/2644208553 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed8005ce0 con 0x7feeec071940 2024-03-08T06:04:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.329+0000 7feee3e00640 1 -- 172.21.15.40:0/2644208553 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed80072f0 con 0x7feeec071940 2024-03-08T06:04:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.329+0000 7feef1c00640 1 -- 172.21.15.40:0/2644208553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec1fc530 con 0x7feeec071940 2024-03-08T06:04:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.329+0000 7feef1c00640 1 -- 172.21.15.40:0/2644208553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec14c390 con 0x7feeec071940 2024-03-08T06:04:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.329+0000 7f47a6000640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f478806b4d0 0x7f478806d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47a81f32c0 tx=0x7f47a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.328+0000 7f3985600640 1 -- 172.21.15.40:0/29750587 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f397c013b90 con 0x7f3970003f10 2024-03-08T06:04:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f7685e00640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f766406b520 0x7f766406d9e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f76881f07b0 tx=0x7f7670009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f3985600640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f396806b520 0x7f396806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f19f8a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19f406de80 0x7f19f40a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f3985600640 1 -- 172.21.15.40:0/29750587 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f397c08be00 con 0x7f3970003f10 2024-03-08T06:04:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f7687200640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] conn(0x7f7664073740 0x7f7664075b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f3985600640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] conn(0x7f39680737a0 0x7f3968075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f3985600640 1 -- 172.21.15.40:0/29750587 --> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3968076270 con 0x7f39680737a0 2024-03-08T06:04:09.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.327+0000 7ff1dfe00640 1 -- 172.21.15.40:0/3082295458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80317c0 con 0x7ff1e015df80 2024-03-08T06:04:09.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.327+0000 7ff1dfe00640 1 -- 172.21.15.40:0/3082295458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d8037070 con 0x7ff1e015df80 2024-03-08T06:04:09.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.328+0000 7ff1dfe00640 1 -- 172.21.15.40:0/3082295458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8031b80 con 0x7ff1e015df80 2024-03-08T06:04:09.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f3985600640 1 -- 172.21.15.40:0/29750587 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f397c08f060 con 0x7f3970003f10 2024-03-08T06:04:09.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7feee1600640 1 -- 172.21.15.40:0/2644208553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feec0001030 con 0x7feeec071940 2024-03-08T06:04:09.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.331+0000 7f398d000640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f396806b520 0x7f396806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.329+0000 7ff1e8600640 1 -- 172.21.15.40:0/3082295458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e015df80 msgr2=0x7ff1e00ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.329+0000 7ff1e8600640 1 --2- 172.21.15.40:0/3082295458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e015df80 0x7ff1e00ce460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00d2180 tx=0x7ff1d802b860 comp rx=0 tx=0).stop 2024-03-08T06:04:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7ff1e8600640 1 -- 172.21.15.40:0/3082295458 shutdown_connections 2024-03-08T06:04:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7ff1e8600640 1 --2- 172.21.15.40:0/3082295458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff1e00b6e30 0x7ff1e00d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7ff1e8600640 1 --2- 172.21.15.40:0/3082295458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e015df80 0x7ff1e00ce460 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7ff1e8600640 1 -- 172.21.15.40:0/3082295458 >> 172.21.15.40:0/3082295458 conn(0x7ff1e006c5e0 msgr2=0x7ff1e006c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.331+0000 7ff1e8600640 1 -- 172.21.15.40:0/3082295458 shutdown_connections 2024-03-08T06:04:09.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.331+0000 7ff1e8600640 1 -- 172.21.15.40:0/3082295458 wait complete. 2024-03-08T06:04:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.332+0000 7f767ea00640 1 -- 172.21.15.40:0/3408334096 <== osd.1 v2:172.21.15.40:6808/981988607 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7680021e50 con 0x7f7664073740 2024-03-08T06:04:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7f398e400640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] conn(0x7f39680737a0 0x7f3968075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7f398d000640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f396806b520 0x7f396806d9e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3974036000 tx=0x7f3974030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.331+0000 7f2b8fe00640 1 -- 172.21.15.40:0/661265619 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b90018020 con 0x7f2b98142ef0 2024-03-08T06:04:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7f2b8fe00640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2b7406b5a0 0x7f2b7406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7f2b8fe00640 1 -- 172.21.15.40:0/661265619 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b9008a500 con 0x7f2b98142ef0 2024-03-08T06:04:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7ff1e8600640 1 Processor -- start 2024-03-08T06:04:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7feee3e00640 1 -- 172.21.15.40:0/2644208553 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feed801a070 con 0x7feeec071940 2024-03-08T06:04:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f19f8a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4071df0 0x7f19f41eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f19f8a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f4154390 0x7f19f41591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f19f8a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40cc430 con 0x7f19f4071df0 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f19f8a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4072220 con 0x7f19f4154390 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.330+0000 7f19f8a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f406e330 con 0x7f19f406de80 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.331+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19f406de80 0x7f19f40a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.331+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19f406de80 0x7f19f40a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:51236/0 (socket says 172.21.15.40:51236) 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.331+0000 7f19f1e00640 1 -- 172.21.15.40:0/2240764791 learned_addr learned my addr 172.21.15.40:0/2240764791 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.332+0000 7f19f2800640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f4154390 0x7f19f41591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7f19f1400640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4071df0 0x7f19f41eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7f19f1e00640 1 -- 172.21.15.40:0/2240764791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f4154390 msgr2=0x7f19f41591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7f19f1e00640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f4154390 0x7f19f41591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7f19f1e00640 1 -- 172.21.15.40:0/2240764791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4071df0 msgr2=0x7f19f41eedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7f19f1e00640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4071df0 0x7f19f41eedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7f19f1e00640 1 -- 172.21.15.40:0/2240764791 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40a8d20 con 0x7f19f406de80 2024-03-08T06:04:09.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.333+0000 7f19f1400640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4071df0 0x7f19f41eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:09.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7f19f1e00640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19f406de80 0x7f19f40a84d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19e4001e80 tx=0x7f19e400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7f2b97400640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2b7406b5a0 0x7f2b7406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7f2b8fe00640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] conn(0x7f2b740737c0 0x7f2b74075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7f2b8fe00640 1 -- 172.21.15.40:0/661265619 --> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b74076290 con 0x7f2b740737c0 2024-03-08T06:04:09.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7feee3e00640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7feecc06b5a0 0x7feecc06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7f2b8fe00640 1 -- 172.21.15.40:0/661265619 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b9008a8f0 con 0x7f2b98142ef0 2024-03-08T06:04:09.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7feee3e00640 1 -- 172.21.15.40:0/2644208553 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feed8089c10 con 0x7feeec071940 2024-03-08T06:04:09.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7f2b8ea00640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] conn(0x7f2b740737c0 0x7f2b74075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.335+0000 7feee3e00640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] conn(0x7feecc0737c0 0x7feecc075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.335+0000 7feeeb400640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7feecc06b5a0 0x7feecc06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.335+0000 7f398e400640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] conn(0x7f39680737a0 0x7f3968075bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7f19ea000640 1 -- 172.21.15.40:0/2240764791 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e40052d0 con 0x7f19f406de80 2024-03-08T06:04:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7f19ea000640 1 -- 172.21.15.40:0/2240764791 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e4005490 con 0x7f19f406de80 2024-03-08T06:04:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7f19ea000640 1 -- 172.21.15.40:0/2240764791 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e40108e0 con 0x7f19f406de80 2024-03-08T06:04:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7f19f8a00640 1 -- 172.21.15.40:0/2240764791 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f40a9000 con 0x7f19f406de80 2024-03-08T06:04:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7f19f8a00640 1 -- 172.21.15.40:0/2240764791 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f40a95c0 con 0x7f19f406de80 2024-03-08T06:04:09.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.335+0000 7f2b97400640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2b7406b5a0 0x7f2b7406da60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b88000f80 tx=0x7f2b8800a000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.334+0000 7ff1e8600640 1 -- start start 2024-03-08T06:04:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.335+0000 7ff1e8600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff1e015df80 0x7ff1e00c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.335+0000 7ff1e8600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e0161ae0 0x7ff1e0071890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.335+0000 7ff1e8600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1e00cacd0 0x7ff1e00cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.335+0000 7ff1e8600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e015e360 con 0x7ff1e0161ae0 2024-03-08T06:04:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.335+0000 7ff1e8600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00cf200 con 0x7ff1e00cacd0 2024-03-08T06:04:09.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.335+0000 7ff1e8600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00d0f10 con 0x7ff1e015df80 2024-03-08T06:04:09.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.335+0000 7feee3e00640 1 -- 172.21.15.40:0/2644208553 --> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feecc076290 con 0x7feecc0737c0 2024-03-08T06:04:09.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7feee3e00640 1 -- 172.21.15.40:0/2644208553 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7feed808a000 con 0x7feeec071940 2024-03-08T06:04:09.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7ff1e5200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e0161ae0 0x7ff1e0071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7ff1e5200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e0161ae0 0x7ff1e0071890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45664/0 (socket says 172.21.15.40:45664) 2024-03-08T06:04:09.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7feef1200640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] conn(0x7feecc0737c0 0x7feecc075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7feeeb400640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7feecc06b5a0 0x7feecc06da60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feedc006a90 tx=0x7feedc006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7f19f8a00640 1 -- 172.21.15.40:0/2240764791 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19cc001030 con 0x7f19f406de80 2024-03-08T06:04:09.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7ff1e5200640 1 -- 172.21.15.40:0/781196500 learned_addr learned my addr 172.21.15.40:0/781196500 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7ff1e5200640 1 -- 172.21.15.40:0/781196500 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff1e015df80 msgr2=0x7ff1e00c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7ff1e5200640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff1e015df80 0x7ff1e00c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7ff1e5200640 1 -- 172.21.15.40:0/781196500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1e00cacd0 msgr2=0x7ff1e00cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7ff1e5200640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1e00cacd0 0x7ff1e00cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.336+0000 7ff1e5200640 1 -- 172.21.15.40:0/781196500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00c8a60 con 0x7ff1e0161ae0 2024-03-08T06:04:09.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.337+0000 7ff1e5200640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e0161ae0 0x7ff1e0071890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00e9b0 tx=0x7ff1cc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.337+0000 7feef1200640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] conn(0x7feecc0737c0 0x7feecc075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.337+0000 7ff1dd600640 1 -- 172.21.15.40:0/781196500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc017070 con 0x7ff1e0161ae0 2024-03-08T06:04:09.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.337+0000 7ff1dd600640 1 -- 172.21.15.40:0/781196500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc00c9a0 con 0x7ff1e0161ae0 2024-03-08T06:04:09.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.337+0000 7ff1e8600640 1 -- 172.21.15.40:0/781196500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e00c8d00 con 0x7ff1e0161ae0 2024-03-08T06:04:09.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.338+0000 7ff1dd600640 1 -- 172.21.15.40:0/781196500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc00cd90 con 0x7ff1e0161ae0 2024-03-08T06:04:09.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.338+0000 7ff1e8600640 1 -- 172.21.15.40:0/781196500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e00c92c0 con 0x7ff1e0161ae0 2024-03-08T06:04:09.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.338+0000 7f3985600640 1 -- 172.21.15.40:0/29750587 <== osd.3 v2:172.21.15.40:6815/405718921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3968076270 con 0x7f39680737a0 2024-03-08T06:04:09.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.339+0000 7ff1e8600640 1 -- 172.21.15.40:0/781196500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff1e0067cb0 con 0x7ff1e0161ae0 2024-03-08T06:04:09.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.337+0000 7f2b8ea00640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] conn(0x7f2b740737c0 0x7f2b74075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.340+0000 7f2b8fe00640 1 -- 172.21.15.40:0/661265619 <== osd.4 v2:172.21.15.53:6816/2858249123 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2b74076290 con 0x7f2b740737c0 2024-03-08T06:04:09.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.340+0000 7feee3e00640 1 -- 172.21.15.40:0/2644208553 <== osd.6 v2:172.21.15.53:6824/2620899738 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7feecc076290 con 0x7feecc0737c0 2024-03-08T06:04:09.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.340+0000 7f19ea000640 1 -- 172.21.15.40:0/2240764791 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19e4010a80 con 0x7f19f406de80 2024-03-08T06:04:09.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.341+0000 7f19ea000640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f19d406b4d0 0x7f19d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.341+0000 7f19ea000640 1 -- 172.21.15.40:0/2240764791 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19e4004ca0 con 0x7f19f406de80 2024-03-08T06:04:09.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.342+0000 7f19ea000640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] conn(0x7f19d4073670 0x7f19d4075a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.342+0000 7f19ea000640 1 -- 172.21.15.40:0/2240764791 --> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f19e4004ec0 con 0x7f19d4073670 2024-03-08T06:04:09.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.342+0000 7f19ea000640 1 -- 172.21.15.40:0/2240764791 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19e408e060 con 0x7f19f406de80 2024-03-08T06:04:09.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.342+0000 7f19f1400640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f19d406b4d0 0x7f19d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.342+0000 7f19f2800640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] conn(0x7f19d4073670 0x7f19d4075a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.341+0000 7ff1dd600640 1 -- 172.21.15.40:0/781196500 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1cc01a070 con 0x7ff1e0161ae0 2024-03-08T06:04:09.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.342+0000 7ff1dd600640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff1c006b520 0x7ff1c006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.342+0000 7ff1dd600640 1 -- 172.21.15.40:0/781196500 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1cc08af80 con 0x7ff1e0161ae0 2024-03-08T06:04:09.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.342+0000 7ff1e5c00640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff1c006b520 0x7ff1c006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.343+0000 7ff1dd600640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] conn(0x7ff1c0073740 0x7ff1c0075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.343+0000 7ff1dd600640 1 -- 172.21.15.40:0/781196500 --> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1c0076210 con 0x7ff1c0073740 2024-03-08T06:04:09.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.343+0000 7ff1dd600640 1 -- 172.21.15.40:0/781196500 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1cc08b330 con 0x7ff1e0161ae0 2024-03-08T06:04:09.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.344+0000 7f19f2800640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] conn(0x7f19d4073670 0x7f19d4075a90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.345+0000 7ff1e6600640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] conn(0x7ff1c0073740 0x7ff1c0075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.345+0000 7f19f1400640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f19d406b4d0 0x7f19d406d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f19ec036000 tx=0x7f19ec031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.346+0000 7f19ea000640 1 -- 172.21.15.40:0/2240764791 <== osd.7 v2:172.21.15.53:6808/597382546 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f19e4004ec0 con 0x7f19d4073670 2024-03-08T06:04:09.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.345+0000 7ff1e5c00640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff1c006b520 0x7ff1c006d9e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8036000 tx=0x7ff1d8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.349+0000 7ff1e6600640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] conn(0x7ff1c0073740 0x7ff1c0075b60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.352+0000 7ff1dd600640 1 -- 172.21.15.40:0/781196500 <== osd.5 v2:172.21.15.53:6800/3704060731 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff1c0076210 con 0x7ff1c0073740 2024-03-08T06:04:09.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.353+0000 7f80ae400640 1 -- 172.21.15.40:0/1967806962 --> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f80a8075090 con 0x7f80840736f0 2024-03-08T06:04:09.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.356+0000 7f768d400640 1 -- 172.21.15.40:0/3408334096 --> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f768806ef30 con 0x7f7664073740 2024-03-08T06:04:09.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.356+0000 7f80a4c00640 1 -- 172.21.15.40:0/1967806962 <== osd.0 v2:172.21.15.40:6800/671313654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f80a8075090 con 0x7f80840736f0 2024-03-08T06:04:09.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.356+0000 7f767ea00640 1 -- 172.21.15.40:0/3408334096 <== osd.1 v2:172.21.15.40:6808/981988607 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f768806ef30 con 0x7f7664073740 2024-03-08T06:04:09.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.356+0000 7f80ae400640 1 -- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] conn(0x7f80840736f0 msgr2=0x7f8084075b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.356+0000 7f80ae400640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] conn(0x7f80840736f0 0x7f8084075b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.357+0000 7f80ae400640 1 -- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f808406b4d0 msgr2=0x7f808406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.357+0000 7f80ae400640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f808406b4d0 0x7f808406d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f80a815dfe0 tx=0x7f80a0009240 comp rx=0 tx=0).stop 2024-03-08T06:04:09.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.357+0000 7f80ae400640 1 -- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b5a20 msgr2=0x7f80a80cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.357+0000 7f80ae400640 1 --2- 172.21.15.40:0/1967806962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b5a20 0x7f80a80cca70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8098009d90 tx=0x7f8098004a40 comp rx=0 tx=0).stop 2024-03-08T06:04:09.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.357+0000 7f80ada00640 1 -- 172.21.15.40:0/1967806962 reap_dead start 2024-03-08T06:04:09.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.357+0000 7f80ae400640 1 -- 172.21.15.40:0/1967806962 shutdown_connections 2024-03-08T06:04:09.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.357+0000 7f80ae400640 1 -- 172.21.15.40:0/1967806962 >> 172.21.15.40:0/1967806962 conn(0x7f80a806c700 msgr2=0x7f80a815fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.357+0000 7f768d400640 1 -- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] conn(0x7f7664073740 msgr2=0x7f7664075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.357+0000 7f768d400640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] conn(0x7f7664073740 0x7f7664075b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.358+0000 7f80ae400640 1 -- 172.21.15.40:0/1967806962 shutdown_connections 2024-03-08T06:04:09.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.358+0000 7f768d400640 1 -- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f766406b520 msgr2=0x7f766406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.358+0000 7f768d400640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f766406b520 0x7f766406d9e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f76881f07b0 tx=0x7f7670009040 comp rx=0 tx=0).stop 2024-03-08T06:04:09.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.358+0000 7f768d400640 1 -- 172.21.15.40:0/3408334096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7688071f90 msgr2=0x7f76880ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.358+0000 7f768d400640 1 --2- 172.21.15.40:0/3408334096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7688071f90 0x7f76880ad020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7680001b70 tx=0x7f7680014420 comp rx=0 tx=0).stop 2024-03-08T06:04:09.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.358+0000 7f7687200640 1 -- 172.21.15.40:0/3408334096 reap_dead start 2024-03-08T06:04:09.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.358+0000 7f80ae400640 1 -- 172.21.15.40:0/1967806962 wait complete. 2024-03-08T06:04:09.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.358+0000 7f768d400640 1 -- 172.21.15.40:0/3408334096 shutdown_connections 2024-03-08T06:04:09.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.358+0000 7f768d400640 1 -- 172.21.15.40:0/3408334096 >> 172.21.15.40:0/3408334096 conn(0x7f768806ae30 msgr2=0x7f76880ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.360+0000 7f768d400640 1 -- 172.21.15.40:0/3408334096 shutdown_connections 2024-03-08T06:04:09.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.360+0000 7f768d400640 1 -- 172.21.15.40:0/3408334096 wait complete. 2024-03-08T06:04:09.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.369+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 --> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f47a8143660 con 0x7f47880736f0 2024-03-08T06:04:09.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.371+0000 7f2b9e800640 1 -- 172.21.15.40:0/661265619 --> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b980b2560 con 0x7f2b740737c0 2024-03-08T06:04:09.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.371+0000 7f479f400640 1 -- 172.21.15.40:0/2094695507 <== osd.2 v2:172.21.15.40:6819/1467838663 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4794037040 con 0x7f47880736f0 2024-03-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.372+0000 7f2b8fe00640 1 -- 172.21.15.40:0/661265619 <== osd.4 v2:172.21.15.53:6816/2858249123 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2b980b2560 con 0x7f2b740737c0 2024-03-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.372+0000 7f397aa00640 1 -- 172.21.15.40:0/29750587 --> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f395c002dd0 con 0x7f39680737a0 2024-03-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.373+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] conn(0x7f47880736f0 msgr2=0x7f4788075b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.373+0000 7f47ade00640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] conn(0x7f47880736f0 0x7f4788075b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.373+0000 7f3985600640 1 -- 172.21.15.40:0/29750587 <== osd.3 v2:172.21.15.40:6815/405718921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f395c002dd0 con 0x7f39680737a0 2024-03-08T06:04:09.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f397aa00640 1 -- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] conn(0x7f39680737a0 msgr2=0x7f3968075bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f397aa00640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] conn(0x7f39680737a0 0x7f3968075bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f478806b4d0 msgr2=0x7f478806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f47ade00640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f478806b4d0 0x7f478806d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47a81f32c0 tx=0x7f47a0009040 comp rx=0 tx=0).stop 2024-03-08T06:04:09.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a806b770 msgr2=0x7f47a8140660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f47ade00640 1 --2- 172.21.15.40:0/2094695507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a806b770 0x7f47a8140660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4798015d40 tx=0x7f4798013560 comp rx=0 tx=0).stop 2024-03-08T06:04:09.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f47a7400640 1 -- 172.21.15.40:0/2094695507 reap_dead start 2024-03-08T06:04:09.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 shutdown_connections 2024-03-08T06:04:09.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 >> 172.21.15.40:0/2094695507 conn(0x7f47a806ac50 msgr2=0x7f47a80b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f397aa00640 1 -- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f396806b520 msgr2=0x7f396806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f397aa00640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f396806b520 0x7f396806d9e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3974036000 tx=0x7f3974030000 comp rx=0 tx=0).stop 2024-03-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f397aa00640 1 -- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970003f10 msgr2=0x7f39881f36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f397aa00640 1 --2- 172.21.15.40:0/29750587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970003f10 0x7f39881f36b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f397c00e4b0 tx=0x7f397c00e970 comp rx=0 tx=0).stop 2024-03-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f2b9e800640 1 -- 172.21.15.40:0/661265619 >> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] conn(0x7f2b740737c0 msgr2=0x7f2b74075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f2b9e800640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] conn(0x7f2b740737c0 0x7f2b74075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f2b9e800640 1 -- 172.21.15.40:0/661265619 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2b7406b5a0 msgr2=0x7f2b7406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f2b9e800640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2b7406b5a0 0x7f2b7406da60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b88000f80 tx=0x7f2b8800a000 comp rx=0 tx=0).stop 2024-03-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f2b9e800640 1 -- 172.21.15.40:0/661265619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b98142ef0 msgr2=0x7f2b981fbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.374+0000 7f2b9e800640 1 --2- 172.21.15.40:0/661265619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b98142ef0 0x7f2b981fbc20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b410 tx=0x7f2b9000b8d0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.375+0000 7f2b97e00640 1 -- 172.21.15.40:0/661265619 reap_dead start 2024-03-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.375+0000 7f398e400640 1 -- 172.21.15.40:0/29750587 reap_dead start 2024-03-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.375+0000 7f2b9e800640 1 -- 172.21.15.40:0/661265619 shutdown_connections 2024-03-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.375+0000 7f2b9e800640 1 -- 172.21.15.40:0/661265619 >> 172.21.15.40:0/661265619 conn(0x7f2b980b16e0 msgr2=0x7f2b9806dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.375+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 shutdown_connections 2024-03-08T06:04:09.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.375+0000 7f47ade00640 1 -- 172.21.15.40:0/2094695507 wait complete. 2024-03-08T06:04:09.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.375+0000 7f2b9e800640 1 -- 172.21.15.40:0/661265619 shutdown_connections 2024-03-08T06:04:09.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.375+0000 7f2b9e800640 1 -- 172.21.15.40:0/661265619 wait complete. 2024-03-08T06:04:09.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.375+0000 7f397aa00640 1 -- 172.21.15.40:0/29750587 shutdown_connections 2024-03-08T06:04:09.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.375+0000 7f397aa00640 1 -- 172.21.15.40:0/29750587 >> 172.21.15.40:0/29750587 conn(0x7f398806ac50 msgr2=0x7f39880cbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.376+0000 7f397aa00640 1 -- 172.21.15.40:0/29750587 shutdown_connections 2024-03-08T06:04:09.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.376+0000 7f397aa00640 1 -- 172.21.15.40:0/29750587 wait complete. 2024-03-08T06:04:09.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.376+0000 7feee1600640 1 -- 172.21.15.40:0/2644208553 --> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feec0002dd0 con 0x7feecc0737c0 2024-03-08T06:04:09.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.378+0000 7feee3e00640 1 -- 172.21.15.40:0/2644208553 <== osd.6 v2:172.21.15.53:6824/2620899738 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7feec0002dd0 con 0x7feecc0737c0 2024-03-08T06:04:09.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.378+0000 7feee1600640 1 -- 172.21.15.40:0/2644208553 >> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] conn(0x7feecc0737c0 msgr2=0x7feecc075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.378+0000 7feee1600640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] conn(0x7feecc0737c0 0x7feecc075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.378+0000 7feee1600640 1 -- 172.21.15.40:0/2644208553 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7feecc06b5a0 msgr2=0x7feecc06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.378+0000 7feee1600640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7feecc06b5a0 0x7feecc06da60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feedc006a90 tx=0x7feedc006a20 comp rx=0 tx=0).stop 2024-03-08T06:04:09.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.378+0000 7feee1600640 1 -- 172.21.15.40:0/2644208553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeec071940 msgr2=0x7feeec145780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.378+0000 7feee1600640 1 --2- 172.21.15.40:0/2644208553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeec071940 0x7feeec145780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feed800c400 tx=0x7feed800c8c0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.379+0000 7feef1200640 1 -- 172.21.15.40:0/2644208553 reap_dead start 2024-03-08T06:04:09.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.379+0000 7feee1600640 1 -- 172.21.15.40:0/2644208553 shutdown_connections 2024-03-08T06:04:09.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.379+0000 7feee1600640 1 -- 172.21.15.40:0/2644208553 >> 172.21.15.40:0/2644208553 conn(0x7feeec0b1590 msgr2=0x7feeec0b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.380+0000 7feee1600640 1 -- 172.21.15.40:0/2644208553 shutdown_connections 2024-03-08T06:04:09.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.381+0000 7f19f8a00640 1 -- 172.21.15.40:0/2240764791 --> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f19cc002dd0 con 0x7f19d4073670 2024-03-08T06:04:09.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.381+0000 7feee1600640 1 -- 172.21.15.40:0/2644208553 wait complete. 2024-03-08T06:04:09.412 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2024-03-08T06:04:09.412 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-08T06:04:09.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.381+0000 7f19ea000640 1 -- 172.21.15.40:0/2240764791 <== osd.7 v2:172.21.15.53:6808/597382546 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f19cc002dd0 con 0x7f19d4073670 2024-03-08T06:04:09.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.382+0000 7f19f8a00640 1 -- 172.21.15.40:0/2240764791 >> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] conn(0x7f19d4073670 msgr2=0x7f19d4075a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.382+0000 7f19f8a00640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] conn(0x7f19d4073670 0x7f19d4075a90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.384+0000 7f19f8a00640 1 -- 172.21.15.40:0/2240764791 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f19d406b4d0 msgr2=0x7f19d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.384+0000 7f19f8a00640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f19d406b4d0 0x7f19d406d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f19ec036000 tx=0x7f19ec031000 comp rx=0 tx=0).stop 2024-03-08T06:04:09.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.384+0000 7f19f8a00640 1 -- 172.21.15.40:0/2240764791 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19f406de80 msgr2=0x7f19f40a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.384+0000 7f19f8a00640 1 --2- 172.21.15.40:0/2240764791 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19f406de80 0x7f19f40a84d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19e4001e80 tx=0x7f19e400c8c0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.384+0000 7f19f2800640 1 -- 172.21.15.40:0/2240764791 reap_dead start 2024-03-08T06:04:09.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.385+0000 7f19f8a00640 1 -- 172.21.15.40:0/2240764791 shutdown_connections 2024-03-08T06:04:09.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.385+0000 7f19f8a00640 1 -- 172.21.15.40:0/2240764791 >> 172.21.15.40:0/2240764791 conn(0x7f19f406ac50 msgr2=0x7f19f415c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.386+0000 7f19f8a00640 1 -- 172.21.15.40:0/2240764791 shutdown_connections 2024-03-08T06:04:09.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.386+0000 7f19f8a00640 1 -- 172.21.15.40:0/2240764791 wait complete. 2024-03-08T06:04:09.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.387+0000 7ff1e8600640 1 -- 172.21.15.40:0/781196500 --> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff1e00d1910 con 0x7ff1c0073740 2024-03-08T06:04:09.416 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2024-03-08T06:04:09.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-08T06:04:09.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.389+0000 7ff1dd600640 1 -- 172.21.15.40:0/781196500 <== osd.5 v2:172.21.15.53:6800/3704060731 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff1e00d1910 con 0x7ff1c0073740 2024-03-08T06:04:09.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.389+0000 7ff1e8600640 1 -- 172.21.15.40:0/781196500 >> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] conn(0x7ff1c0073740 msgr2=0x7ff1c0075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.389+0000 7ff1e8600640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] conn(0x7ff1c0073740 0x7ff1c0075b60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.390+0000 7ff1e8600640 1 -- 172.21.15.40:0/781196500 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff1c006b520 msgr2=0x7ff1c006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.390+0000 7ff1e8600640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff1c006b520 0x7ff1c006d9e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8036000 tx=0x7ff1d8030000 comp rx=0 tx=0).stop 2024-03-08T06:04:09.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.390+0000 7ff1e8600640 1 -- 172.21.15.40:0/781196500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e0161ae0 msgr2=0x7ff1e0071890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.390+0000 7ff1e8600640 1 --2- 172.21.15.40:0/781196500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e0161ae0 0x7ff1e0071890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00e9b0 tx=0x7ff1cc00ee70 comp rx=0 tx=0).stop 2024-03-08T06:04:09.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.390+0000 7ff1e6600640 1 -- 172.21.15.40:0/781196500 reap_dead start 2024-03-08T06:04:09.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.390+0000 7ff1e8600640 1 -- 172.21.15.40:0/781196500 shutdown_connections 2024-03-08T06:04:09.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.390+0000 7ff1e8600640 1 -- 172.21.15.40:0/781196500 >> 172.21.15.40:0/781196500 conn(0x7ff1e006c5e0 msgr2=0x7ff1e00b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.390+0000 7ff1e8600640 1 -- 172.21.15.40:0/781196500 shutdown_connections 2024-03-08T06:04:09.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.390+0000 7ff1e8600640 1 -- 172.21.15.40:0/781196500 wait complete. 2024-03-08T06:04:09.419 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-03-08T06:04:09.419 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-08T06:04:09.420 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-03-08T06:04:09.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-08T06:04:09.421 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-03-08T06:04:09.421 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-08T06:04:09.422 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-03-08T06:04:09.422 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-08T06:04:09.423 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2024-03-08T06:04:09.423 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-08T06:04:09.424 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-03-08T06:04:09.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-08T06:04:09.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.756+0000 7f7867e00640 1 Processor -- start 2024-03-08T06:04:09.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.757+0000 7f76b3200640 1 Processor -- start 2024-03-08T06:04:09.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.757+0000 7f7867e00640 1 -- start start 2024-03-08T06:04:09.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.757+0000 7f76b3200640 1 -- start start 2024-03-08T06:04:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.759+0000 7f7867e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7868162150 0x7f78681630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.759+0000 7f7867e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7868099bc0 0x7f78680c4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.759+0000 7f7867e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f786809a040 0x7f78680c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.759+0000 7f7867e00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7868165c20 con 0x7f786809a040 2024-03-08T06:04:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.759+0000 7f7867e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78681635f0 con 0x7f7868162150 2024-03-08T06:04:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.759+0000 7f7867e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78680b28f0 con 0x7f7868099bc0 2024-03-08T06:04:09.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.760+0000 7f7866a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7868162150 0x7f78681630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.760+0000 7f7867400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f786809a040 0x7f78680c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.760+0000 7f7866000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7868099bc0 0x7f78680c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.760+0000 7f7866000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7868099bc0 0x7f78680c4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:51244/0 (socket says 172.21.15.40:51244) 2024-03-08T06:04:09.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.760+0000 7f7866000640 1 -- 172.21.15.40:0/2817159442 learned_addr learned my addr 172.21.15.40:0/2817159442 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.760+0000 7f3941e00640 1 Processor -- start 2024-03-08T06:04:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.761+0000 7f3941e00640 1 -- start start 2024-03-08T06:04:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.761+0000 7f76b3200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac0b27a0 0x7f76ac0d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.761+0000 7f76b1e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f76ac1611f0 0x7f76ac0b54e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60908/0 (socket says 172.21.15.40:60908) 2024-03-08T06:04:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.761+0000 7f7866a00640 1 -- 172.21.15.40:0/2817159442 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7868099bc0 msgr2=0x7f78680c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.761+0000 7f7866a00640 1 --2- 172.21.15.40:0/2817159442 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7868099bc0 0x7f78680c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.761+0000 7f7866a00640 1 -- 172.21.15.40:0/2817159442 >> v2:172.21.15.40:3300/0 conn(0x7f786809a040 msgr2=0x7f78680c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.761+0000 7f7866a00640 1 --2- 172.21.15.40:0/2817159442 >> v2:172.21.15.40:3300/0 conn(0x7f786809a040 0x7f78680c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.761+0000 7f76b1e00640 1 -- 172.21.15.40:0/7482363 learned_addr learned my addr 172.21.15.40:0/7482363 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.761+0000 7f7866a00640 1 -- 172.21.15.40:0/2817159442 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786806bf00 con 0x7f7868162150 2024-03-08T06:04:09.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.762+0000 7f7867400640 1 --2- 172.21.15.40:0/2817159442 >> v2:172.21.15.40:3300/0 conn(0x7f786809a040 0x7f78680c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:09.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.762+0000 7f76b2800640 1 --2- 172.21.15.40:0/7482363 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac0b27a0 0x7f76ac0d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.762+0000 7f76b3200640 1 -- 172.21.15.40:0/7482363 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ac0d3410 con 0x7f76ac1611f0 2024-03-08T06:04:09.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.762+0000 7f76b3200640 1 -- 172.21.15.40:0/7482363 --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ac0d36f0 con 0x7f76ac0b5bf0 2024-03-08T06:04:09.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.762+0000 7f76b3200640 1 -- 172.21.15.40:0/7482363 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0ceac0 con 0x7f76ac0b27a0 2024-03-08T06:04:09.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.763+0000 7f3941e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 0x7f393c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.763+0000 7f3941e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f393c06cc10 con 0x7f393c15e840 2024-03-08T06:04:09.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.763+0000 7f3941e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f393c15ec90 con 0x7f393c06c7d0 2024-03-08T06:04:09.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.763+0000 7f3941e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0b96c0 con 0x7f393c15dd50 2024-03-08T06:04:09.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.763+0000 7f76b0a00640 1 -- 172.21.15.40:0/7482363 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425570332 0 0) 0x7f76ac0d36f0 con 0x7f76ac0b5bf0 2024-03-08T06:04:09.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.763+0000 7f76b0a00640 1 -- 172.21.15.40:0/7482363 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7694003520 con 0x7f76ac0b5bf0 2024-03-08T06:04:09.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.763+0000 7f76b0a00640 1 -- 172.21.15.40:0/7482363 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644202459 0 0) 0x7f76ac0d3410 con 0x7f76ac1611f0 2024-03-08T06:04:09.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.763+0000 7f76b0a00640 1 -- 172.21.15.40:0/7482363 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76ac0d36f0 con 0x7f76ac1611f0 2024-03-08T06:04:09.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.764+0000 7f76b2800640 1 -- 172.21.15.40:0/7482363 >> v1:172.21.15.53:6789/0 conn(0x7f76ac0b5bf0 legacy=0x7f76ac0d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.764+0000 7f76b2800640 1 -- 172.21.15.40:0/7482363 >> v1:172.21.15.40:6789/0 conn(0x7f76ac1611f0 legacy=0x7f76ac0b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.764+0000 7f76b2800640 1 -- 172.21.15.40:0/7482363 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac0ced10 con 0x7f76ac0b27a0 2024-03-08T06:04:09.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.764+0000 7f7866a00640 1 --2- 172.21.15.40:0/2817159442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7868162150 0x7f78681630b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78680b7940 tx=0x7f785402b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca0b63b83b555aec server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.764+0000 7f7865600640 1 -- 172.21.15.40:0/2817159442 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854038070 con 0x7f7868162150 2024-03-08T06:04:09.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.765+0000 7f7865600640 1 -- 172.21.15.40:0/2817159442 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7854031700 con 0x7f7868162150 2024-03-08T06:04:09.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.765+0000 7f7865600640 1 -- 172.21.15.40:0/2817159442 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854031ad0 con 0x7f7868162150 2024-03-08T06:04:09.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.763+0000 7f393a000640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f393c06c7d0 0x7f393c15e130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.40:38734/0 (socket says 172.21.15.40:38734) 2024-03-08T06:04:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.764+0000 7f393a000640 1 -- 172.21.15.40:0/496638588 learned_addr learned my addr 172.21.15.40:0/496638588 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.764+0000 7f393aa00640 1 --2- 172.21.15.40:0/496638588 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 0x7f393c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.765+0000 7f393aa00640 1 -- 172.21.15.40:0/496638588 >> v1:172.21.15.53:6789/0 conn(0x7f393c06c7d0 legacy=0x7f393c15e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.765+0000 7f393aa00640 1 -- 172.21.15.40:0/496638588 >> v1:172.21.15.40:6789/0 conn(0x7f393c15e840 legacy=0x7f393c0d2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.766+0000 7f7867e00640 1 -- 172.21.15.40:0/2817159442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7868162150 msgr2=0x7f78681630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.766+0000 7f7867e00640 1 --2- 172.21.15.40:0/2817159442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7868162150 0x7f78681630b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78680b7940 tx=0x7f785402b7a0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.765+0000 7f393aa00640 1 -- 172.21.15.40:0/496638588 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c15f220 con 0x7f393c15dd50 2024-03-08T06:04:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.766+0000 7f76b2800640 1 --2- 172.21.15.40:0/7482363 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac0b27a0 0x7f76ac0d0ff0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009dc0 tx=0x7f76a802be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d502457dbeeac1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.766+0000 7f7867e00640 1 -- 172.21.15.40:0/2817159442 shutdown_connections 2024-03-08T06:04:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.766+0000 7f7867e00640 1 --2- 172.21.15.40:0/2817159442 >> v2:172.21.15.40:3300/0 conn(0x7f786809a040 0x7f78680c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.766+0000 7f7867e00640 1 --2- 172.21.15.40:0/2817159442 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7868099bc0 0x7f78680c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.766+0000 7f7867e00640 1 --2- 172.21.15.40:0/2817159442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7868162150 0x7f78681630b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.766+0000 7f7867e00640 1 -- 172.21.15.40:0/2817159442 >> 172.21.15.40:0/2817159442 conn(0x7f78680d1890 msgr2=0x7f78680d1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.767+0000 7f7867e00640 1 -- 172.21.15.40:0/2817159442 shutdown_connections 2024-03-08T06:04:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.767+0000 7f7867e00640 1 -- 172.21.15.40:0/2817159442 wait complete. 2024-03-08T06:04:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.767+0000 7f76b0a00640 1 -- 172.21.15.40:0/7482363 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8039070 con 0x7f76ac0b27a0 2024-03-08T06:04:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.767+0000 7f76b0a00640 1 -- 172.21.15.40:0/7482363 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a802cbb0 con 0x7f76ac0b27a0 2024-03-08T06:04:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.767+0000 7f76b0a00640 1 -- 172.21.15.40:0/7482363 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8034440 con 0x7f76ac0b27a0 2024-03-08T06:04:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.768+0000 7f76b3200640 1 -- 172.21.15.40:0/7482363 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac0b27a0 msgr2=0x7f76ac0d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.768+0000 7f76b3200640 1 --2- 172.21.15.40:0/7482363 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac0b27a0 0x7f76ac0d0ff0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009dc0 tx=0x7f76a802be00 comp rx=0 tx=0).stop 2024-03-08T06:04:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.767+0000 7f393aa00640 1 --2- 172.21.15.40:0/496638588 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 0x7f393c06c290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f39340097d0 tx=0x7f393402b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df9f71b33b5f7db server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.767+0000 7f798f200640 1 Processor -- start 2024-03-08T06:04:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.768+0000 7f76b3200640 1 -- 172.21.15.40:0/7482363 shutdown_connections 2024-03-08T06:04:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.768+0000 7f76b3200640 1 --2- 172.21.15.40:0/7482363 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac0b27a0 0x7f76ac0d0ff0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.768+0000 7f76b3200640 1 -- 172.21.15.40:0/7482363 >> 172.21.15.40:0/7482363 conn(0x7f76ac06c700 msgr2=0x7f76ac06cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.768+0000 7f798f200640 1 -- start start 2024-03-08T06:04:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.768+0000 7f76b3200640 1 -- 172.21.15.40:0/7482363 shutdown_connections 2024-03-08T06:04:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.768+0000 7f76b3200640 1 -- 172.21.15.40:0/7482363 wait complete. 2024-03-08T06:04:09.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.769+0000 7f3939600640 1 -- 172.21.15.40:0/496638588 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934037070 con 0x7f393c15dd50 2024-03-08T06:04:09.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.769+0000 7f3939600640 1 -- 172.21.15.40:0/496638588 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3934030710 con 0x7f393c15dd50 2024-03-08T06:04:09.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.769+0000 7f3939600640 1 -- 172.21.15.40:0/496638588 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393402be60 con 0x7f393c15dd50 2024-03-08T06:04:09.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.770+0000 7f798f200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f79901611f0 0x7f79900b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.770+0000 7f798f200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79900b5bf0 0x7f79900d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.770+0000 7f798f200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79900d3410 con 0x7f79900d0ff0 2024-03-08T06:04:09.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.770+0000 7f798f200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79900d1480 con 0x7f79900b5bf0 2024-03-08T06:04:09.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.770+0000 7f798f200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79900ceac0 con 0x7f79901611f0 2024-03-08T06:04:09.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.770+0000 7f7867e00640 1 Processor -- start 2024-03-08T06:04:09.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f3941e00640 1 -- 172.21.15.40:0/496638588 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 msgr2=0x7f393c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.770+0000 7f798e800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79900b5bf0 0x7f79900d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f3941e00640 1 --2- 172.21.15.40:0/496638588 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 0x7f393c06c290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f39340097d0 tx=0x7f393402b660 comp rx=0 tx=0).stop 2024-03-08T06:04:09.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f76b3200640 1 Processor -- start 2024-03-08T06:04:09.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f3941e00640 1 -- 172.21.15.40:0/496638588 shutdown_connections 2024-03-08T06:04:09.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f3941e00640 1 --2- 172.21.15.40:0/496638588 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 0x7f393c06c290 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f3941e00640 1 -- 172.21.15.40:0/496638588 >> 172.21.15.40:0/496638588 conn(0x7f393c066fc0 msgr2=0x7f393c0d3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f7867e00640 1 -- start start 2024-03-08T06:04:09.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.770+0000 7f798e800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79900b5bf0 0x7f79900d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45668/0 (socket says 172.21.15.40:45668) 2024-03-08T06:04:09.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f798e800640 1 -- 172.21.15.40:0/329786576 learned_addr learned my addr 172.21.15.40:0/329786576 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f798de00640 1 --2- 172.21.15.40:0/329786576 >> v2:172.21.15.53:3300/0 conn(0x7f79901611f0 0x7f79900b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f76b3200640 1 -- start start 2024-03-08T06:04:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f76b3200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76ac0b27a0 0x7f76ac146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f76b3200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ac0b5bf0 0x7f76ac15c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f76b3200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac1611f0 0x7f76ac15cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f76b3200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0d1e40 con 0x7f76ac0b27a0 2024-03-08T06:04:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f76b3200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0cf080 con 0x7f76ac0b5bf0 2024-03-08T06:04:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f76b3200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0d1fc0 con 0x7f76ac1611f0 2024-03-08T06:04:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f3941e00640 1 -- 172.21.15.40:0/496638588 shutdown_connections 2024-03-08T06:04:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305999882 0 0) 0x7f79900d3410 con 0x7f79900d0ff0 2024-03-08T06:04:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7974003520 con 0x7f79900d0ff0 2024-03-08T06:04:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 999681851 0 0) 0x7f7974003520 con 0x7f79900d0ff0 2024-03-08T06:04:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 >> v2:172.21.15.53:3300/0 conn(0x7f79901611f0 msgr2=0x7f79900b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f798ca00640 1 --2- 172.21.15.40:0/329786576 >> v2:172.21.15.53:3300/0 conn(0x7f79901611f0 0x7f79900b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 >> v2:172.21.15.40:3300/0 conn(0x7f79900b5bf0 msgr2=0x7f79900d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f798ca00640 1 --2- 172.21.15.40:0/329786576 >> v2:172.21.15.40:3300/0 conn(0x7f79900b5bf0 0x7f79900d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79900d36f0 con 0x7f79900d0ff0 2024-03-08T06:04:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f7980001670 con 0x7f79900d0ff0 2024-03-08T06:04:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f94f3e00640 1 Processor -- start 2024-03-08T06:04:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f94f3e00640 1 -- start start 2024-03-08T06:04:09.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.771+0000 7f3941e00640 1 -- 172.21.15.40:0/496638588 wait complete. 2024-03-08T06:04:09.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.772+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 >> v1:172.21.15.40:6789/0 conn(0x7f79900d0ff0 legacy=0x7f79900b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f798ca00640 1 --2- 172.21.15.40:0/329786576 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7974003a90 0x7f7974003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f798ca00640 1 --2- 172.21.15.40:0/329786576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 0x7f7974004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f798ca00640 1 --2- 172.21.15.40:0/329786576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79740051c0 0x7f797400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79900ceac0 con 0x7f79740051c0 2024-03-08T06:04:09.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797400c0f0 con 0x7f7974004800 2024-03-08T06:04:09.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797400c2f0 con 0x7f7974003a90 2024-03-08T06:04:09.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f76b2800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac1611f0 0x7f76ac15cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f76b2800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac1611f0 0x7f76ac15cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:51282/0 (socket says 172.21.15.40:51282) 2024-03-08T06:04:09.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f090c800640 1 Processor -- start 2024-03-08T06:04:09.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f76b2800640 1 -- 172.21.15.40:0/3279999069 learned_addr learned my addr 172.21.15.40:0/3279999069 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f3941e00640 1 Processor -- start 2024-03-08T06:04:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f798de00640 1 --2- 172.21.15.40:0/329786576 >> v2:172.21.15.53:3300/0 conn(0x7f79901611f0 0x7f79900b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.773+0000 7f798e800640 1 --2- 172.21.15.40:0/329786576 >> v2:172.21.15.40:3300/0 conn(0x7f79900b5bf0 0x7f79900d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f798d400640 1 --2- 172.21.15.40:0/329786576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 0x7f7974004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f798de00640 1 --2- 172.21.15.40:0/329786576 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7974003a90 0x7f7974003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f76b1400640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ac0b5bf0 0x7f76ac15c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.775+0000 7f76b1e00640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76ac0b27a0 0x7f76ac146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.775+0000 7f76b2800640 1 -- 172.21.15.40:0/3279999069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ac0b5bf0 msgr2=0x7f76ac15c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.775+0000 7f76b2800640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ac0b5bf0 0x7f76ac15c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.775+0000 7f76b2800640 1 -- 172.21.15.40:0/3279999069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76ac0b27a0 msgr2=0x7f76ac146990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.775+0000 7f76b2800640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76ac0b27a0 0x7f76ac146990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.775+0000 7f76b2800640 1 -- 172.21.15.40:0/3279999069 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac1e8980 con 0x7f76ac1611f0 2024-03-08T06:04:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.776+0000 7f76b1400640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ac0b5bf0 0x7f76ac15c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f798e800640 1 --2- 172.21.15.40:0/329786576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79740051c0 0x7f797400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f94f3e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ec147690 0x7f94ec1622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.775+0000 7f94f3e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec162800 con 0x7f94ec15efb0 2024-03-08T06:04:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.775+0000 7f94f3e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec0b4ca0 con 0x7f94ec071f10 2024-03-08T06:04:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.775+0000 7f94f3e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec15e4d0 con 0x7f94ec147690 2024-03-08T06:04:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.775+0000 7f94f1e00640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f94ec071f10 0x7f94ec0ae050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.40:38736/0 (socket says 172.21.15.40:38736) 2024-03-08T06:04:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.776+0000 7f94f1e00640 1 -- 172.21.15.40:0/3850727623 learned_addr learned my addr 172.21.15.40:0/3850727623 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.776+0000 7f94f1400640 1 --2- 172.21.15.40:0/3850727623 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ec147690 0x7f94ec1622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f090c800640 1 -- start start 2024-03-08T06:04:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f798d400640 1 -- 172.21.15.40:0/329786576 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7974003a90 msgr2=0x7f7974003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f798d400640 1 --2- 172.21.15.40:0/329786576 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7974003a90 0x7f7974003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f798d400640 1 -- 172.21.15.40:0/329786576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79740051c0 msgr2=0x7f797400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f798d400640 1 --2- 172.21.15.40:0/329786576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79740051c0 0x7f797400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f798d400640 1 -- 172.21.15.40:0/329786576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79900d36f0 con 0x7f7974004800 2024-03-08T06:04:09.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.776+0000 7f798e800640 1 -- 172.21.15.40:0/329786576 reap_dead start 2024-03-08T06:04:09.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.776+0000 7f94ebe00640 1 -- 172.21.15.40:0/3850727623 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099456034 0 0) 0x7f94e0001110 con 0x7f94ec15efb0 2024-03-08T06:04:09.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f76b2800640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac1611f0 0x7f76ac15cab0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f76a802c410 tx=0x7f76a8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f94ebe00640 1 -- 172.21.15.40:0/3850727623 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94d4003520 con 0x7f94ec15efb0 2024-03-08T06:04:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f7867e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7868099bc0 0x7f78680cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.774+0000 7f3941e00640 1 -- start start 2024-03-08T06:04:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.775+0000 7f3941e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f393c06c7d0 0x7f393c1f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.776+0000 7f3941e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 0x7f393c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.776+0000 7f3941e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f393c15e840 0x7f393c1f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.776+0000 7f3941e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0d1f40 con 0x7f393c06c7d0 2024-03-08T06:04:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.776+0000 7f3941e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0d1c60 con 0x7f393c15e840 2024-03-08T06:04:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.776+0000 7f3941e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0d2220 con 0x7f393c15dd50 2024-03-08T06:04:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f393aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f393c06c7d0 0x7f393c1f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f393aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f393c06c7d0 0x7f393c1f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45702/0 (socket says 172.21.15.40:45702) 2024-03-08T06:04:09.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f798d400640 1 --2- 172.21.15.40:0/329786576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 0x7f7974004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7980001eb0 tx=0x7f7974005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8aee7e363721db server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f76a6000640 1 -- 172.21.15.40:0/3279999069 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8044020 con 0x7f76ac1611f0 2024-03-08T06:04:09.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f76a6000640 1 -- 172.21.15.40:0/3279999069 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a8035e50 con 0x7f76ac1611f0 2024-03-08T06:04:09.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f76a6000640 1 -- 172.21.15.40:0/3279999069 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8039040 con 0x7f76ac1611f0 2024-03-08T06:04:09.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.778+0000 7f76b3200640 1 -- 172.21.15.40:0/3279999069 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ac15ad60 con 0x7f76ac1611f0 2024-03-08T06:04:09.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f393aa00640 1 -- 172.21.15.40:0/1549950302 learned_addr learned my addr 172.21.15.40:0/1549950302 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f7867e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f786809a040 0x7f78680cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f7867e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7868162150 0x7f78680cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f7867e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78680b2550 con 0x7f7868099bc0 2024-03-08T06:04:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f7867e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786806db80 con 0x7f7868162150 2024-03-08T06:04:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f7867e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78680b26d0 con 0x7f786809a040 2024-03-08T06:04:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.778+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980007ca0 con 0x7f7974004800 2024-03-08T06:04:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f393a000640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 0x7f393c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.778+0000 7f393b400640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f393c15e840 0x7f393c1f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.778+0000 7f7866a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7868099bc0 0x7f78680cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.778+0000 7f76b3200640 1 -- 172.21.15.40:0/3279999069 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ac1e8c20 con 0x7f76ac1611f0 2024-03-08T06:04:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f7866000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f786809a040 0x7f78680cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f7866a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7868099bc0 0x7f78680cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45710/0 (socket says 172.21.15.40:45710) 2024-03-08T06:04:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f94f1400640 1 -- 172.21.15.40:0/3850727623 >> v1:172.21.15.53:6789/0 conn(0x7f94ec071f10 legacy=0x7f94ec0ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.777+0000 7f94f1400640 1 -- 172.21.15.40:0/3850727623 >> v1:172.21.15.40:6789/0 conn(0x7f94ec15efb0 legacy=0x7f94ec071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.778+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7980030070 con 0x7f7974004800 2024-03-08T06:04:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.778+0000 7f94f1400640 1 -- 172.21.15.40:0/3850727623 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec15e7b0 con 0x7f94ec147690 2024-03-08T06:04:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f798f200640 1 -- 172.21.15.40:0/329786576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 msgr2=0x7f7974004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f7866a00640 1 -- 172.21.15.40:0/1729168535 learned_addr learned my addr 172.21.15.40:0/1729168535 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f798ca00640 1 -- 172.21.15.40:0/329786576 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980007ca0 con 0x7f7974004800 2024-03-08T06:04:09.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f798f200640 1 --2- 172.21.15.40:0/329786576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 0x7f7974004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7980001eb0 tx=0x7f7974005830 comp rx=0 tx=0).stop 2024-03-08T06:04:09.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f393a000640 1 -- 172.21.15.40:0/1549950302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f393c15e840 msgr2=0x7f393c1f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f393a000640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f393c15e840 0x7f393c1f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f393a000640 1 -- 172.21.15.40:0/1549950302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f393c06c7d0 msgr2=0x7f393c1f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f393a000640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f393c06c7d0 0x7f393c1f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f393a000640 1 -- 172.21.15.40:0/1549950302 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c1f3650 con 0x7f393c15dd50 2024-03-08T06:04:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f393aa00640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f393c06c7d0 0x7f393c1f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.779+0000 7f7866a00640 1 -- 172.21.15.40:0/1729168535 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f786809a040 msgr2=0x7f78680cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f7866a00640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f786809a040 0x7f78680cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f94f1400640 1 --2- 172.21.15.40:0/3850727623 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ec147690 0x7f94ec1622c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009dc0 tx=0x7f94e402bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fca2c437d1793577 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f090c800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0904154390 0x7f090415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f798f200640 1 -- 172.21.15.40:0/329786576 shutdown_connections 2024-03-08T06:04:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f798f200640 1 --2- 172.21.15.40:0/329786576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 0x7f7974004c80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f798f200640 1 -- 172.21.15.40:0/329786576 >> 172.21.15.40:0/329786576 conn(0x7f799006c700 msgr2=0x7f799006cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f393a000640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 0x7f393c0cbb80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f392400c4d0 tx=0x7f392400c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f798f200640 1 -- 172.21.15.40:0/329786576 shutdown_connections 2024-03-08T06:04:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f7866a00640 1 -- 172.21.15.40:0/1729168535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7868162150 msgr2=0x7f78680cc670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T06:04:09.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f76b3200640 1 -- 172.21.15.40:0/3279999069 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ac075090 con 0x7f76ac1611f0 2024-03-08T06:04:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f94ebe00640 1 -- 172.21.15.40:0/3850727623 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4039070 con 0x7f94ec147690 2024-03-08T06:04:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f7867400640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7868162150 0x7f78680cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f7866a00640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7868162150 0x7f78680cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f7866a00640 1 -- 172.21.15.40:0/1729168535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680c8dc0 con 0x7f7868099bc0 2024-03-08T06:04:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f94ebe00640 1 -- 172.21.15.40:0/3850727623 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e402c770 con 0x7f94ec147690 2024-03-08T06:04:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f94ebe00640 1 -- 172.21.15.40:0/3850727623 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4039070 con 0x7f94ec147690 2024-03-08T06:04:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f0909400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f090406de80 0x7f09040718b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60952/0 (socket says 172.21.15.40:60952) 2024-03-08T06:04:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f7866000640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f786809a040 0x7f78680cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f7867400640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7868162150 0x7f78680cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f0909e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0904154390 0x7f090415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f3933400640 1 -- 172.21.15.40:0/1549950302 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924018070 con 0x7f393c15dd50 2024-03-08T06:04:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f3933400640 1 -- 172.21.15.40:0/1549950302 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39240074e0 con 0x7f393c15dd50 2024-03-08T06:04:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f3933400640 1 -- 172.21.15.40:0/1549950302 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39240049e0 con 0x7f393c15dd50 2024-03-08T06:04:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f3941e00640 1 -- 172.21.15.40:0/1549950302 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c0cc3d0 con 0x7f393c15dd50 2024-03-08T06:04:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f090c800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090415b2f0 con 0x7f090406de80 2024-03-08T06:04:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f090c800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090406d3e0 con 0x7f0904071fc0 2024-03-08T06:04:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f090c800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090406d9f0 con 0x7f0904154390 2024-03-08T06:04:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f0909400640 1 -- 172.21.15.40:0/58223005 learned_addr learned my addr 172.21.15.40:0/58223005 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f3941e00640 1 -- 172.21.15.40:0/1549950302 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c0cc990 con 0x7f393c15dd50 2024-03-08T06:04:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f94f3e00640 1 -- 172.21.15.40:0/3850727623 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ec147690 msgr2=0x7f94ec1622c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f7866a00640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7868099bc0 0x7f78680cd9e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7854000c00 tx=0x7f78540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.781+0000 7f94f3e00640 1 --2- 172.21.15.40:0/3850727623 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ec147690 0x7f94ec1622c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009dc0 tx=0x7f94e402bc40 comp rx=0 tx=0).stop 2024-03-08T06:04:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.782+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262918402 0 0) 0x7f08f4001110 con 0x7f090406de80 2024-03-08T06:04:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.782+0000 7f785ea00640 1 -- 172.21.15.40:0/1729168535 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854042070 con 0x7f7868099bc0 2024-03-08T06:04:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.782+0000 7f7867e00640 1 -- 172.21.15.40:0/1729168535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78680c8ae0 con 0x7f7868099bc0 2024-03-08T06:04:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.782+0000 7f785ea00640 1 -- 172.21.15.40:0/1729168535 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785403d870 con 0x7f7868099bc0 2024-03-08T06:04:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.782+0000 7f7867e00640 1 -- 172.21.15.40:0/1729168535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78680c9710 con 0x7f7868099bc0 2024-03-08T06:04:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.780+0000 7f798f200640 1 -- 172.21.15.40:0/329786576 wait complete. 2024-03-08T06:04:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.782+0000 7f94f3e00640 1 -- 172.21.15.40:0/3850727623 shutdown_connections 2024-03-08T06:04:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.782+0000 7f94f3e00640 1 --2- 172.21.15.40:0/3850727623 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ec147690 0x7f94ec1622c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.782+0000 7f94f3e00640 1 -- 172.21.15.40:0/3850727623 >> 172.21.15.40:0/3850727623 conn(0x7f94ec06ac50 msgr2=0x7f94ec06b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08ec003520 con 0x7f090406de80 2024-03-08T06:04:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f94f3e00640 1 -- 172.21.15.40:0/3850727623 shutdown_connections 2024-03-08T06:04:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2960953905 0 0) 0x7f08ec003520 con 0x7f090406de80 2024-03-08T06:04:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 >> v2:172.21.15.53:3300/0 conn(0x7f0904154390 msgr2=0x7f090415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 --2- 172.21.15.40:0/58223005 >> v2:172.21.15.53:3300/0 conn(0x7f0904154390 0x7f090415adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 >> v1:172.21.15.53:6789/0 conn(0x7f0904071fc0 legacy=0x7f09040b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040cc760 con 0x7f090406de80 2024-03-08T06:04:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f08f4001840 con 0x7f090406de80 2024-03-08T06:04:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f94f3e00640 1 -- 172.21.15.40:0/3850727623 wait complete. 2024-03-08T06:04:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.784+0000 7f785ea00640 1 -- 172.21.15.40:0/1729168535 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854042070 con 0x7f7868099bc0 2024-03-08T06:04:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 >> v1:172.21.15.40:6789/0 conn(0x7f090406de80 legacy=0x7f09040718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 --2- 172.21.15.40:0/58223005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 0x7f08ec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 --2- 172.21.15.40:0/58223005 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08ec004800 0x7f08ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 --2- 172.21.15.40:0/58223005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08ec0051c0 0x7f08ec00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090406d9f0 con 0x7f08ec0051c0 2024-03-08T06:04:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0104d0 con 0x7f08ec003a00 2024-03-08T06:04:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.783+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0099b0 con 0x7f08ec004800 2024-03-08T06:04:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.784+0000 7f0909e00640 1 --2- 172.21.15.40:0/58223005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 0x7f08ec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.784+0000 7f090a800640 1 --2- 172.21.15.40:0/58223005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08ec0051c0 0x7f08ec00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.784+0000 7f0909400640 1 --2- 172.21.15.40:0/58223005 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08ec004800 0x7f08ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.784+0000 7f798f200640 1 Processor -- start 2024-03-08T06:04:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.784+0000 7f0909e00640 1 -- 172.21.15.40:0/58223005 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08ec004800 msgr2=0x7f08ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.785+0000 7f7867e00640 1 -- 172.21.15.40:0/1729168535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7840005290 con 0x7f7868099bc0 2024-03-08T06:04:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.784+0000 7f0909e00640 1 --2- 172.21.15.40:0/58223005 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08ec004800 0x7f08ec004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.784+0000 7f0909e00640 1 -- 172.21.15.40:0/58223005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08ec0051c0 msgr2=0x7f08ec00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.785+0000 7f0909e00640 1 --2- 172.21.15.40:0/58223005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08ec0051c0 0x7f08ec00fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.785+0000 7f090a800640 1 -- 172.21.15.40:0/58223005 reap_dead start 2024-03-08T06:04:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.785+0000 7f0909e00640 1 -- 172.21.15.40:0/58223005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec003520 con 0x7f08ec003a00 2024-03-08T06:04:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.785+0000 7f798f200640 1 -- start start 2024-03-08T06:04:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.786+0000 7f94f3e00640 1 Processor -- start 2024-03-08T06:04:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.787+0000 7f3941e00640 1 -- 172.21.15.40:0/1549950302 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393c0cfc40 con 0x7f393c15dd50 2024-03-08T06:04:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.785+0000 7f76a6000640 1 -- 172.21.15.40:0/3279999069 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76a8069070 con 0x7f76ac1611f0 2024-03-08T06:04:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.787+0000 7f798f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 0x7f79900cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.787+0000 7f76a6000640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f769006b4d0 0x7f769006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.787+0000 7f798f200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f79900cdf20 0x7f799015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.787+0000 7f76a6000640 1 -- 172.21.15.40:0/3279999069 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76a80ad770 con 0x7f76ac1611f0 2024-03-08T06:04:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.787+0000 7f798f200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79900b7d70 0x7f79900b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.786+0000 7f0909e00640 1 --2- 172.21.15.40:0/58223005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 0x7f08ec003e20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009ac0 tx=0x7f08ec005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921c5fde4144f33c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.787+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8039070 con 0x7f08ec003a00 2024-03-08T06:04:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.787+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f802c6b0 con 0x7f08ec003a00 2024-03-08T06:04:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.787+0000 7f0908a00640 1 -- 172.21.15.40:0/58223005 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8009690 con 0x7f08ec003a00 2024-03-08T06:04:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f798f200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79901615d0 con 0x7f79900b7d70 2024-03-08T06:04:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f798f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79900cf080 con 0x7f7974004800 2024-03-08T06:04:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f798f200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7990161750 con 0x7f79900cdf20 2024-03-08T06:04:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.786+0000 7f94f3e00640 1 -- start start 2024-03-08T06:04:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.787+0000 7f94f3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec071f10 0x7f94ec15d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f94f3e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ec147690 0x7f94ec1463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f94f3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ec15efb0 0x7f94ec146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f94f3e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0b4880 con 0x7f94ec071f10 2024-03-08T06:04:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f94f3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0b4700 con 0x7f94ec15efb0 2024-03-08T06:04:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f94f3e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0b4a00 con 0x7f94ec147690 2024-03-08T06:04:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f76b1e00640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f769006b4d0 0x7f769006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f798de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 0x7f79900cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f798de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 0x7f79900cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:49702/0 (socket says 172.21.15.40:49702) 2024-03-08T06:04:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f798de00640 1 -- 172.21.15.40:0/620686028 learned_addr learned my addr 172.21.15.40:0/620686028 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f798d400640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f79900cdf20 0x7f799015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f798e800640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79900b7d70 0x7f79900b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f090c800640 1 -- 172.21.15.40:0/58223005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 msgr2=0x7f08ec003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f090c800640 1 --2- 172.21.15.40:0/58223005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 0x7f08ec003e20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009ac0 tx=0x7f08ec005830 comp rx=0 tx=0).stop 2024-03-08T06:04:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.788+0000 7f785ea00640 1 -- 172.21.15.40:0/1729168535 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7854045070 con 0x7f7868099bc0 2024-03-08T06:04:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.789+0000 7f798de00640 1 -- 172.21.15.40:0/620686028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f79900cdf20 msgr2=0x7f799015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.789+0000 7f798de00640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f79900cdf20 0x7f799015cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.789+0000 7f798de00640 1 -- 172.21.15.40:0/620686028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79900b7d70 msgr2=0x7f79900b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.789+0000 7f798de00640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79900b7d70 0x7f79900b81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.789+0000 7f798de00640 1 -- 172.21.15.40:0/620686028 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799006eba0 con 0x7f7974004800 2024-03-08T06:04:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.789+0000 7f798e800640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79900b7d70 0x7f79900b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.789+0000 7f94f1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ec15efb0 0x7f94ec146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.789+0000 7f94f1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ec15efb0 0x7f94ec146930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:49714/0 (socket says 172.21.15.40:49714) 2024-03-08T06:04:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.789+0000 7f94f1e00640 1 -- 172.21.15.40:0/1993186108 learned_addr learned my addr 172.21.15.40:0/1993186108 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.789+0000 7f785ea00640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f784806b8d0 0x7f784806dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.789+0000 7f785ea00640 1 -- 172.21.15.40:0/1729168535 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f785403d670 con 0x7f7868099bc0 2024-03-08T06:04:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f798de00640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 0x7f79900cc0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f797c011860 tx=0x7f797c011c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f090c800640 1 -- 172.21.15.40:0/58223005 shutdown_connections 2024-03-08T06:04:09.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f090c800640 1 --2- 172.21.15.40:0/58223005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 0x7f08ec003e20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f090c800640 1 -- 172.21.15.40:0/58223005 >> 172.21.15.40:0/58223005 conn(0x7f090406ac50 msgr2=0x7f090406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f76b1e00640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f769006b4d0 0x7f769006d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76a0007fd0 tx=0x7f76a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f94f1400640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec071f10 0x7f94ec15d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f7866000640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f784806b8d0 0x7f784806dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f94f1e00640 1 -- 172.21.15.40:0/1993186108 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ec147690 msgr2=0x7f94ec1463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f94f1e00640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ec147690 0x7f94ec1463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f94f1e00640 1 -- 172.21.15.40:0/1993186108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec071f10 msgr2=0x7f94ec15d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f94f1e00640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec071f10 0x7f94ec15d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f94f1e00640 1 -- 172.21.15.40:0/1993186108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec139010 con 0x7f94ec15efb0 2024-03-08T06:04:09.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f7986000640 1 -- 172.21.15.40:0/620686028 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c004230 con 0x7f7974004800 2024-03-08T06:04:09.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f7986000640 1 -- 172.21.15.40:0/620686028 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c0043d0 con 0x7f7974004800 2024-03-08T06:04:09.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.791+0000 7f7986000640 1 -- 172.21.15.40:0/620686028 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c00f9e0 con 0x7f7974004800 2024-03-08T06:04:09.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.791+0000 7f798f200640 1 -- 172.21.15.40:0/620686028 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79900cc730 con 0x7f7974004800 2024-03-08T06:04:09.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f090c800640 1 -- 172.21.15.40:0/58223005 shutdown_connections 2024-03-08T06:04:09.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.790+0000 7f090c800640 1 -- 172.21.15.40:0/58223005 wait complete. 2024-03-08T06:04:09.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.791+0000 7f798f200640 1 -- 172.21.15.40:0/620686028 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79900cccf0 con 0x7f7974004800 2024-03-08T06:04:09.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.791+0000 7f94f1e00640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ec15efb0 0x7f94ec146930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94d800c780 tx=0x7f94d800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.791+0000 7f8689400640 1 Processor -- start 2024-03-08T06:04:09.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.791+0000 7f8689400640 1 -- start start 2024-03-08T06:04:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.792+0000 7f94e9600640 1 -- 172.21.15.40:0/1993186108 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8018070 con 0x7f94ec15efb0 2024-03-08T06:04:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.792+0000 7f94e9600640 1 -- 172.21.15.40:0/1993186108 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d80072d0 con 0x7f94ec15efb0 2024-03-08T06:04:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.792+0000 7f94e9600640 1 -- 172.21.15.40:0/1993186108 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d80047e0 con 0x7f94ec15efb0 2024-03-08T06:04:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.792+0000 7f94f3e00640 1 -- 172.21.15.40:0/1993186108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec141430 con 0x7f94ec15efb0 2024-03-08T06:04:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.792+0000 7f94f3e00640 1 -- 172.21.15.40:0/1993186108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec141150 con 0x7f94ec15efb0 2024-03-08T06:04:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.793+0000 7f798f200640 1 -- 172.21.15.40:0/620686028 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7990075090 con 0x7f7974004800 2024-03-08T06:04:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.793+0000 7fb2fca00640 1 Processor -- start 2024-03-08T06:04:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.793+0000 7f7866000640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f784806b8d0 0x7f784806dd90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f78680c8470 tx=0x7f7858008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.793+0000 7fb2fca00640 1 -- start start 2024-03-08T06:04:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.793+0000 7f8689400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f86840d09e0 0x7f86841665f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.793+0000 7f8689400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 0x7f86840d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.793+0000 7f8689400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8684162700 con 0x7f86840d0dc0 2024-03-08T06:04:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.793+0000 7f8689400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8684166b30 con 0x7f86840d09e0 2024-03-08T06:04:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.793+0000 7f8689400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86841635a0 con 0x7f86840d1840 2024-03-08T06:04:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.794+0000 7f090c800640 1 Processor -- start 2024-03-08T06:04:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.794+0000 7f94f3e00640 1 -- 172.21.15.40:0/1993186108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec067cb0 con 0x7f94ec15efb0 2024-03-08T06:04:09.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.794+0000 7f090c800640 1 -- start start 2024-03-08T06:04:09.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.794+0000 7f8688a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 0x7f86840d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.794+0000 7f8688a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 0x7f86840d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:51344/0 (socket says 172.21.15.40:51344) 2024-03-08T06:04:09.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.794+0000 7f8688a00640 1 -- 172.21.15.40:0/3827156794 learned_addr learned my addr 172.21.15.40:0/3827156794 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.795+0000 7f7986000640 1 -- 172.21.15.40:0/620686028 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f797c00fb80 con 0x7f7974004800 2024-03-08T06:04:09.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.794+0000 7f8683e00640 1 --2- 172.21.15.40:0/3827156794 >> v2:172.21.15.53:3300/0 conn(0x7f86840d09e0 0x7f86841665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.795+0000 7f785ea00640 1 -- 172.21.15.40:0/1729168535 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7854030070 con 0x7f7868099bc0 2024-03-08T06:04:09.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.795+0000 7f8688a00640 1 -- 172.21.15.40:0/3827156794 >> v2:172.21.15.53:3300/0 conn(0x7f86840d09e0 msgr2=0x7f86841665f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.795+0000 7f8688a00640 1 --2- 172.21.15.40:0/3827156794 >> v2:172.21.15.53:3300/0 conn(0x7f86840d09e0 0x7f86841665f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.795+0000 7f8688a00640 1 -- 172.21.15.40:0/3827156794 >> v1:172.21.15.53:6789/0 conn(0x7f86840d0dc0 legacy=0x7f8684162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.795+0000 7f8688a00640 1 -- 172.21.15.40:0/3827156794 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840ce430 con 0x7f86840d1840 2024-03-08T06:04:09.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.795+0000 7fb2fca00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb2f815be20 0x7fb2f80b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.795+0000 7fb2fca00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2f80cfb70 0x7fb2f80cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.795+0000 7fb2fca00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f809ad40 con 0x7fb2f80cc480 2024-03-08T06:04:09.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.795+0000 7fb2fca00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f80d0510 con 0x7fb2f80cfb70 2024-03-08T06:04:09.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.795+0000 7fb2fca00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80cf920 con 0x7fb2f815be20 2024-03-08T06:04:09.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7f090c800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 0x7f09041379e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7fb2f5400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2f80cfb70 0x7fb2f80cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7fb2f5e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb2f815be20 0x7fb2f80b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7fb2f6800640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb2f80cc480 0x7fb2f80cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.40:38756/0 (socket says 172.21.15.40:38756) 2024-03-08T06:04:09.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7fb2f5400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2f80cfb70 0x7fb2f80cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45734/0 (socket says 172.21.15.40:45734) 2024-03-08T06:04:09.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7fb2f5400640 1 -- 172.21.15.40:0/2513868506 learned_addr learned my addr 172.21.15.40:0/2513868506 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7f090c800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0904156400 0x7f09040b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7f090c800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f09040b0810 0x7f0904139060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7f090c800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040cc430 con 0x7f0904156400 2024-03-08T06:04:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7f090c800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0904154770 con 0x7f08ec003a00 2024-03-08T06:04:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7f090c800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090406e330 con 0x7f09040b0810 2024-03-08T06:04:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7f7986000640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f797006b4d0 0x7f797006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7fb2f5400640 1 -- 172.21.15.40:0/2513868506 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb2f815be20 msgr2=0x7fb2f80b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7fb2f5400640 1 --2- 172.21.15.40:0/2513868506 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb2f815be20 0x7fb2f80b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7f7986000640 1 -- 172.21.15.40:0/620686028 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f797c08af30 con 0x7f7974004800 2024-03-08T06:04:09.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7f0909e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 0x7f09041379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7fb2f5400640 1 -- 172.21.15.40:0/2513868506 >> v1:172.21.15.53:6789/0 conn(0x7fb2f80cc480 legacy=0x7fb2f80cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:09.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7f0909e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 0x7f09041379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:49726/0 (socket says 172.21.15.40:49726) 2024-03-08T06:04:09.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7f0909e00640 1 -- 172.21.15.40:0/2020733551 learned_addr learned my addr 172.21.15.40:0/2020733551 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.796+0000 7f94e9600640 1 -- 172.21.15.40:0/1993186108 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94d8004980 con 0x7f94ec15efb0 2024-03-08T06:04:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7f090a800640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f09040b0810 0x7f0904139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7f798d400640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f797006b4d0 0x7f797006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7f94e9600640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f94cc06b4d0 0x7f94cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7f0909400640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0904156400 0x7f09040b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7f8688a00640 1 --2- 172.21.15.40:0/3827156794 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 0x7f86840d1c50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f867c009e50 tx=0x7f867c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18278f51b9c6be22 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.797+0000 7fb2f5400640 1 -- 172.21.15.40:0/2513868506 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f809b020 con 0x7fb2f80cfb70 2024-03-08T06:04:09.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.798+0000 7f94e9600640 1 -- 172.21.15.40:0/1993186108 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94d8089780 con 0x7f94ec15efb0 2024-03-08T06:04:09.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.798+0000 7fb2f5e00640 1 --2- 172.21.15.40:0/2513868506 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb2f815be20 0x7fb2f80b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:09.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.798+0000 7f94f1400640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f94cc06b4d0 0x7f94cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.798+0000 7f0909e00640 1 -- 172.21.15.40:0/2020733551 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f09040b0810 msgr2=0x7f0904139060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.798+0000 7f0909e00640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f09040b0810 0x7f0904139060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.798+0000 7f0909e00640 1 -- 172.21.15.40:0/2020733551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0904156400 msgr2=0x7f09040b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.798+0000 7f0909e00640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0904156400 0x7f09040b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.798+0000 7f0909e00640 1 -- 172.21.15.40:0/2020733551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0904137f20 con 0x7f08ec003a00 2024-03-08T06:04:09.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.798+0000 7f0909400640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0904156400 0x7f09040b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:09.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.798+0000 7fb2f5400640 1 --2- 172.21.15.40:0/2513868506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2f80cfb70 0x7fb2f80cffd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009e50 tx=0x7fb2f002bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88284557f74bb41 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.799+0000 7f798d400640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f797006b4d0 0x7f797006d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7980035000 tx=0x7f7980009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.799+0000 7f94f1400640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f94cc06b4d0 0x7f94cc06d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f94e4036000 tx=0x7f94e4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.799+0000 7fb2f4a00640 1 -- 172.21.15.40:0/2513868506 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f002caa0 con 0x7fb2f80cfb70 2024-03-08T06:04:09.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.799+0000 7fb2f4a00640 1 -- 172.21.15.40:0/2513868506 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f002cc40 con 0x7fb2f80cfb70 2024-03-08T06:04:09.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.799+0000 7fb2f4a00640 1 -- 172.21.15.40:0/2513868506 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0034610 con 0x7fb2f80cfb70 2024-03-08T06:04:09.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.799+0000 7f0909e00640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 0x7f09041379e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009be0 tx=0x7f08f8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.799+0000 7f76a6000640 1 -- 172.21.15.40:0/3279999069 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76a804e030 con 0x7f76ac1611f0 2024-03-08T06:04:09.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.800+0000 7fb2fca00640 1 -- 172.21.15.40:0/2513868506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2f80cfb70 msgr2=0x7fb2f80cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.800+0000 7fb2fca00640 1 --2- 172.21.15.40:0/2513868506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2f80cfb70 0x7fb2f80cffd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009e50 tx=0x7fb2f002bf00 comp rx=0 tx=0).stop 2024-03-08T06:04:09.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.800+0000 7f8682a00640 1 -- 172.21.15.40:0/3827156794 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c02ca70 con 0x7f86840d1840 2024-03-08T06:04:09.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.800+0000 7fb2fca00640 1 -- 172.21.15.40:0/2513868506 shutdown_connections 2024-03-08T06:04:09.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.800+0000 7fb2fca00640 1 --2- 172.21.15.40:0/2513868506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2f80cfb70 0x7fb2f80cffd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.800+0000 7fb2fca00640 1 --2- 172.21.15.40:0/2513868506 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb2f815be20 0x7fb2f80b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.800+0000 7f8682a00640 1 -- 172.21.15.40:0/3827156794 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c02cc10 con 0x7f86840d1840 2024-03-08T06:04:09.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.800+0000 7f8682a00640 1 -- 172.21.15.40:0/3827156794 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c0343f0 con 0x7f86840d1840 2024-03-08T06:04:09.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.800+0000 7fb2fca00640 1 -- 172.21.15.40:0/2513868506 >> 172.21.15.40:0/2513868506 conn(0x7fb2f80b3460 msgr2=0x7fb2f80b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.800+0000 7f8689400640 1 -- 172.21.15.40:0/3827156794 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 msgr2=0x7f86840d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.801+0000 7fb2fca00640 1 -- 172.21.15.40:0/2513868506 shutdown_connections 2024-03-08T06:04:09.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.800+0000 7f8689400640 1 --2- 172.21.15.40:0/3827156794 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 0x7f86840d1c50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f867c009e50 tx=0x7f867c02bed0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.801+0000 7f8689400640 1 -- 172.21.15.40:0/3827156794 shutdown_connections 2024-03-08T06:04:09.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.801+0000 7f8689400640 1 --2- 172.21.15.40:0/3827156794 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 0x7f86840d1c50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.801+0000 7f8689400640 1 --2- 172.21.15.40:0/3827156794 >> v2:172.21.15.53:3300/0 conn(0x7f86840d09e0 0x7f86841665f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.801+0000 7f8689400640 1 -- 172.21.15.40:0/3827156794 >> 172.21.15.40:0/3827156794 conn(0x7f868406c700 msgr2=0x7f868406cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:09.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.801+0000 7f8689400640 1 -- 172.21.15.40:0/3827156794 shutdown_connections 2024-03-08T06:04:09.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.801+0000 7f8689400640 1 -- 172.21.15.40:0/3827156794 wait complete. 2024-03-08T06:04:09.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.801+0000 7fb2fca00640 1 -- 172.21.15.40:0/2513868506 wait complete. 2024-03-08T06:04:09.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.803+0000 7f3933400640 1 -- 172.21.15.40:0/1549950302 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f392401b070 con 0x7f393c15dd50 2024-03-08T06:04:09.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.803+0000 7f8689400640 1 Processor -- start 2024-03-08T06:04:09.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.803+0000 7f8689400640 1 -- start start 2024-03-08T06:04:09.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.803+0000 7fb2fca00640 1 Processor -- start 2024-03-08T06:04:09.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.804+0000 7fb2fca00640 1 -- start start 2024-03-08T06:04:09.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.804+0000 7f3933400640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f391806b520 0x7f391806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.804+0000 7f3933400640 1 -- 172.21.15.40:0/1549950302 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3924089ab0 con 0x7f393c15dd50 2024-03-08T06:04:09.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.805+0000 7f393aa00640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f391806b520 0x7f391806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.805+0000 7f8689400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86840d09e0 0x7f8684150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.805+0000 7f8689400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86840d0dc0 0x7f86840b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.805+0000 7f8689400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 0x7f86840b8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.805+0000 7f8689400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840d37d0 con 0x7f86840d09e0 2024-03-08T06:04:09.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.805+0000 7f8689400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684162970 con 0x7f86840d0dc0 2024-03-08T06:04:09.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.805+0000 7f8689400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840d1250 con 0x7f86840d1840 2024-03-08T06:04:09.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.805+0000 7f8683e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86840d09e0 0x7f8684150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.805+0000 7f8683e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86840d09e0 0x7f8684150ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45748/0 (socket says 172.21.15.40:45748) 2024-03-08T06:04:09.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.805+0000 7f8683e00640 1 -- 172.21.15.40:0/283560522 learned_addr learned my addr 172.21.15.40:0/283560522 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.806+0000 7f8688a00640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 0x7f86840b8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.806+0000 7f393aa00640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f391806b520 0x7f391806d9e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3934000c00 tx=0x7f3934008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.806+0000 7f8683e00640 1 -- 172.21.15.40:0/283560522 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 msgr2=0x7f86840b8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.806+0000 7f8683e00640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 0x7f86840b8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.806+0000 7f8683e00640 1 -- 172.21.15.40:0/283560522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86840d0dc0 msgr2=0x7f86840b80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.806+0000 7f8683e00640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86840d0dc0 0x7f86840b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.806+0000 7f8683e00640 1 -- 172.21.15.40:0/283560522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840b74a0 con 0x7f86840d09e0 2024-03-08T06:04:09.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.806+0000 7f8683e00640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86840d09e0 0x7f8684150ca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f867400e9e0 tx=0x7f867400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.807+0000 7fb2fca00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb2f80cc480 0x7fb2f81f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.807+0000 7f867be00640 1 -- 172.21.15.40:0/283560522 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674017070 con 0x7f86840d09e0 2024-03-08T06:04:09.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.807+0000 7f867be00640 1 -- 172.21.15.40:0/283560522 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867400c9d0 con 0x7f86840d09e0 2024-03-08T06:04:09.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.807+0000 7f867be00640 1 -- 172.21.15.40:0/283560522 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867400cdc0 con 0x7f86840d09e0 2024-03-08T06:04:09.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.807+0000 7fb2fca00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2f80cfb70 0x7fb2f81e5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.807+0000 7fb2fca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f815be20 0x7fb2f80c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.807+0000 7f8689400640 1 -- 172.21.15.40:0/283560522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86840b3660 con 0x7f86840d09e0 2024-03-08T06:04:09.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.807+0000 7f8689400640 1 -- 172.21.15.40:0/283560522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86840b3c20 con 0x7f86840d09e0 2024-03-08T06:04:09.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.808+0000 7fb2f5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2f80cfb70 0x7fb2f81e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.808+0000 7fb2fca00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80cf6c0 con 0x7fb2f80cfb70 2024-03-08T06:04:09.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.808+0000 7fb2fca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80cf540 con 0x7fb2f815be20 2024-03-08T06:04:09.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.808+0000 7fb2fca00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80cd6b0 con 0x7fb2f80cc480 2024-03-08T06:04:09.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.808+0000 7fb2f5e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb2f80cc480 0x7fb2f81f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.808+0000 7f8689400640 1 -- 172.21.15.40:0/283560522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8684075090 con 0x7f86840d09e0 2024-03-08T06:04:09.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.808+0000 7fb2f5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2f80cfb70 0x7fb2f81e5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45760/0 (socket says 172.21.15.40:45760) 2024-03-08T06:04:09.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.808+0000 7fb2f5400640 1 -- 172.21.15.40:0/1929395716 learned_addr learned my addr 172.21.15.40:0/1929395716 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.808+0000 7fb2f6800640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f815be20 0x7fb2f80c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.809+0000 7fb2f5400640 1 -- 172.21.15.40:0/1929395716 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb2f80cc480 msgr2=0x7fb2f81f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.809+0000 7fb2f5400640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb2f80cc480 0x7fb2f81f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.809+0000 7fb2f5400640 1 -- 172.21.15.40:0/1929395716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f815be20 msgr2=0x7fb2f80c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.809+0000 7fb2f5400640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f815be20 0x7fb2f80c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.809+0000 7fb2f5400640 1 -- 172.21.15.40:0/1929395716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f80b5950 con 0x7fb2f80cfb70 2024-03-08T06:04:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.809+0000 7fb2f5400640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2f80cfb70 0x7fb2f81e5570 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0036040 tx=0x7fb2f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.809+0000 7f867be00640 1 -- 172.21.15.40:0/283560522 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f867401a070 con 0x7f86840d09e0 2024-03-08T06:04:09.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.810+0000 7f867be00640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f866406b4d0 0x7f866406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.810+0000 7f867be00640 1 -- 172.21.15.40:0/283560522 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f867408a670 con 0x7f86840d09e0 2024-03-08T06:04:09.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.810+0000 7f8683400640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f866406b4d0 0x7f866406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.811+0000 7f8683400640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f866406b4d0 0x7f866406d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f868415ce80 tx=0x7f86700079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.813+0000 7f7986000640 1 -- 172.21.15.40:0/620686028 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f797c0b88c0 con 0x7f7974004800 2024-03-08T06:04:09.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.815+0000 7fb2ee000640 1 -- 172.21.15.40:0/1929395716 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0043020 con 0x7fb2f80cfb70 2024-03-08T06:04:09.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.815+0000 7fb2ee000640 1 -- 172.21.15.40:0/1929395716 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f003e450 con 0x7fb2f80cfb70 2024-03-08T06:04:09.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.815+0000 7f867be00640 1 -- 172.21.15.40:0/283560522 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8674059450 con 0x7f86840d09e0 2024-03-08T06:04:09.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.815+0000 7fb2ee000640 1 -- 172.21.15.40:0/1929395716 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0009510 con 0x7fb2f80cfb70 2024-03-08T06:04:09.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.815+0000 7fb2fca00640 1 -- 172.21.15.40:0/1929395716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f80b5c30 con 0x7fb2f80cfb70 2024-03-08T06:04:09.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.815+0000 7f3933400640 1 -- 172.21.15.40:0/1549950302 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f393c0cfc40 con 0x7f393c15dd50 2024-03-08T06:04:09.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.815+0000 7fb2fca00640 1 -- 172.21.15.40:0/1929395716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f80b61f0 con 0x7fb2f80cfb70 2024-03-08T06:04:09.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.817+0000 7fb2fca00640 1 -- 172.21.15.40:0/1929395716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f80d13e0 con 0x7fb2f80cfb70 2024-03-08T06:04:09.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.819+0000 7fb2ee000640 1 -- 172.21.15.40:0/1929395716 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2f0046020 con 0x7fb2f80cfb70 2024-03-08T06:04:09.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.820+0000 7fb2ee000640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb2d806b5a0 0x7fb2d806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.820+0000 7fb2ee000640 1 -- 172.21.15.40:0/1929395716 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2f00acc10 con 0x7fb2f80cfb70 2024-03-08T06:04:09.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.821+0000 7fb2f5e00640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb2d806b5a0 0x7fb2d806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.822+0000 7fb2f5e00640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb2d806b5a0 0x7fb2d806da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8006fd0 tx=0x7fb2e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.827+0000 7f08fe000640 1 -- 172.21.15.40:0/2020733551 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8042020 con 0x7f08ec003a00 2024-03-08T06:04:09.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.827+0000 7f08fe000640 1 -- 172.21.15.40:0/2020733551 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f802de20 con 0x7f08ec003a00 2024-03-08T06:04:09.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.827+0000 7f08fe000640 1 -- 172.21.15.40:0/2020733551 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8039040 con 0x7f08ec003a00 2024-03-08T06:04:09.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.827+0000 7f090c800640 1 -- 172.21.15.40:0/2020733551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09041553a0 con 0x7f08ec003a00 2024-03-08T06:04:09.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.827+0000 7f94e9600640 1 -- 172.21.15.40:0/1993186108 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94ec141430 con 0x7f94ec15efb0 2024-03-08T06:04:09.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.828+0000 7fb2ee000640 1 -- 172.21.15.40:0/1929395716 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2f80d13e0 con 0x7fb2f80cfb70 2024-03-08T06:04:09.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.828+0000 7f090c800640 1 -- 172.21.15.40:0/2020733551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0904155920 con 0x7f08ec003a00 2024-03-08T06:04:09.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.830+0000 7f090c800640 1 -- 172.21.15.40:0/2020733551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09040b4ef0 con 0x7f08ec003a00 2024-03-08T06:04:09.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.832+0000 7f08fe000640 1 -- 172.21.15.40:0/2020733551 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08f8045020 con 0x7f08ec003a00 2024-03-08T06:04:09.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.834+0000 7f08fe000640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f08e806b5a0 0x7f08e806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:09.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.834+0000 7f08fe000640 1 -- 172.21.15.40:0/2020733551 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08f80ab1e0 con 0x7f08ec003a00 2024-03-08T06:04:09.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.834+0000 7f0909400640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f08e806b5a0 0x7f08e806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:09.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.836+0000 7f0909400640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f08e806b5a0 0x7f08e806da60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08f40019b0 tx=0x7f08f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:09.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:09.847+0000 7f08fe000640 1 -- 172.21.15.40:0/2020733551 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09040b4ef0 con 0x7f08ec003a00 2024-03-08T06:04:10.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.210+0000 7f7867e00640 1 -- 172.21.15.40:0/1729168535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7840005850 con 0x7f7868099bc0 2024-03-08T06:04:10.211 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2024-03-08T06:04:10.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.211+0000 7f785ea00640 1 -- 172.21.15.40:0/1729168535 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f785407a880 con 0x7f7868099bc0 2024-03-08T06:04:10.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.214+0000 7f7867e00640 1 -- 172.21.15.40:0/1729168535 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f784806b8d0 msgr2=0x7f784806dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.214+0000 7f7867e00640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f784806b8d0 0x7f784806dd90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f78680c8470 tx=0x7f7858008040 comp rx=0 tx=0).stop 2024-03-08T06:04:10.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.214+0000 7f7867e00640 1 -- 172.21.15.40:0/1729168535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7868099bc0 msgr2=0x7f78680cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.215+0000 7f7867e00640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7868099bc0 0x7f78680cd9e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7854000c00 tx=0x7f78540046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.216+0000 7f7867e00640 1 -- 172.21.15.40:0/1729168535 shutdown_connections 2024-03-08T06:04:10.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.216+0000 7f7867e00640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f784806b8d0 0x7f784806dd90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.216+0000 7f7867e00640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7868162150 0x7f78680cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.216+0000 7f7867e00640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f786809a040 0x7f78680cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.216+0000 7f7867e00640 1 --2- 172.21.15.40:0/1729168535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7868099bc0 0x7f78680cd9e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.216+0000 7f7867e00640 1 -- 172.21.15.40:0/1729168535 >> 172.21.15.40:0/1729168535 conn(0x7f78680d1890 msgr2=0x7f78680b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:10.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.217+0000 7f7867e00640 1 -- 172.21.15.40:0/1729168535 shutdown_connections 2024-03-08T06:04:10.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.217+0000 7f7867e00640 1 -- 172.21.15.40:0/1729168535 wait complete. 2024-03-08T06:04:10.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.228+0000 7f76b3200640 1 -- 172.21.15.40:0/3279999069 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f76ac06b280 con 0x7f76ac1611f0 2024-03-08T06:04:10.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.230+0000 7f76a6000640 1 -- 172.21.15.40:0/3279999069 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f76a807c680 con 0x7f76ac1611f0 2024-03-08T06:04:10.232 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2024-03-08T06:04:10.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.233+0000 7f76b3200640 1 -- 172.21.15.40:0/3279999069 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f769006b4d0 msgr2=0x7f769006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.233+0000 7f76b3200640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f769006b4d0 0x7f769006d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76a0007fd0 tx=0x7f76a0009040 comp rx=0 tx=0).stop 2024-03-08T06:04:10.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.233+0000 7f76b3200640 1 -- 172.21.15.40:0/3279999069 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac1611f0 msgr2=0x7f76ac15cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.233+0000 7f76b3200640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac1611f0 0x7f76ac15cab0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f76a802c410 tx=0x7f76a8004a40 comp rx=0 tx=0).stop 2024-03-08T06:04:10.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.234+0000 7f76b3200640 1 -- 172.21.15.40:0/3279999069 shutdown_connections 2024-03-08T06:04:10.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.234+0000 7f76b3200640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f769006b4d0 0x7f769006d990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.234+0000 7f76b3200640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f76ac1611f0 0x7f76ac15cab0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.234+0000 7f76b3200640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ac0b5bf0 0x7f76ac15c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.234+0000 7f76b3200640 1 --2- 172.21.15.40:0/3279999069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76ac0b27a0 0x7f76ac146990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.234+0000 7f76b3200640 1 -- 172.21.15.40:0/3279999069 >> 172.21.15.40:0/3279999069 conn(0x7f76ac06c700 msgr2=0x7f76ac0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:10.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.234+0000 7f76b3200640 1 -- 172.21.15.40:0/3279999069 shutdown_connections 2024-03-08T06:04:10.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.234+0000 7f76b3200640 1 -- 172.21.15.40:0/3279999069 wait complete. 2024-03-08T06:04:10.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-03-08T06:04:10.243 DEBUG:teuthology.parallel:result is None 2024-03-08T06:04:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.246+0000 7f8689400640 1 -- 172.21.15.40:0/283560522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f86840d31e0 con 0x7f86840d09e0 2024-03-08T06:04:10.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.247+0000 7f867be00640 1 -- 172.21.15.40:0/283560522 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8674058d50 con 0x7f86840d09e0 2024-03-08T06:04:10.247 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-03-08T06:04:10.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.250+0000 7f798f200640 1 -- 172.21.15.40:0/620686028 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7990160770 con 0x7f7974004800 2024-03-08T06:04:10.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.251+0000 7f8689400640 1 -- 172.21.15.40:0/283560522 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f866406b4d0 msgr2=0x7f866406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.251+0000 7f8689400640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f866406b4d0 0x7f866406d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f868415ce80 tx=0x7f86700079b0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.251+0000 7f8689400640 1 -- 172.21.15.40:0/283560522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86840d09e0 msgr2=0x7f8684150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.251+0000 7f8689400640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86840d09e0 0x7f8684150ca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f867400e9e0 tx=0x7f867400eea0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.252+0000 7f8689400640 1 -- 172.21.15.40:0/283560522 shutdown_connections 2024-03-08T06:04:10.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.252+0000 7f8689400640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f866406b4d0 0x7f866406d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.252+0000 7f8689400640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f86840d1840 0x7f86840b8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.252+0000 7f8689400640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86840d0dc0 0x7f86840b80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.252+0000 7f8689400640 1 --2- 172.21.15.40:0/283560522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86840d09e0 0x7f8684150ca0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.252+0000 7f8689400640 1 -- 172.21.15.40:0/283560522 >> 172.21.15.40:0/283560522 conn(0x7f868406c700 msgr2=0x7f8684163920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:10.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.252+0000 7f8689400640 1 -- 172.21.15.40:0/283560522 shutdown_connections 2024-03-08T06:04:10.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.252+0000 7f8689400640 1 -- 172.21.15.40:0/283560522 wait complete. 2024-03-08T06:04:10.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.253+0000 7f7986000640 1 -- 172.21.15.40:0/620686028 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f797c019070 con 0x7f7974004800 2024-03-08T06:04:10.253 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-03-08T06:04:10.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.254+0000 7f3941e00640 1 -- 172.21.15.40:0/1549950302 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f393c0b8f10 con 0x7f393c15dd50 2024-03-08T06:04:10.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.255+0000 7f3933400640 1 -- 172.21.15.40:0/1549950302 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f393c1f3650 con 0x7f393c15dd50 2024-03-08T06:04:10.256 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-03-08T06:04:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.256+0000 7f798f200640 1 -- 172.21.15.40:0/620686028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f797006b4d0 msgr2=0x7f797006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.256+0000 7f798f200640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f797006b4d0 0x7f797006d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7980035000 tx=0x7f7980009030 comp rx=0 tx=0).stop 2024-03-08T06:04:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.256+0000 7f798f200640 1 -- 172.21.15.40:0/620686028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 msgr2=0x7f79900cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.256+0000 7f798f200640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 0x7f79900cc0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f797c011860 tx=0x7f797c011c60 comp rx=0 tx=0).stop 2024-03-08T06:04:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 -- 172.21.15.40:0/1549950302 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f391806b520 msgr2=0x7f391806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f391806b520 0x7f391806d9e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3934000c00 tx=0x7f3934008030 comp rx=0 tx=0).stop 2024-03-08T06:04:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 -- 172.21.15.40:0/1549950302 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 msgr2=0x7f393c0cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 0x7f393c0cbb80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f392400c4d0 tx=0x7f392400c990 comp rx=0 tx=0).stop 2024-03-08T06:04:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f798f200640 1 -- 172.21.15.40:0/620686028 shutdown_connections 2024-03-08T06:04:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f798f200640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f797006b4d0 0x7f797006d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f798f200640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79900b7d70 0x7f79900b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f798f200640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f79900cdf20 0x7f799015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f798f200640 1 --2- 172.21.15.40:0/620686028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7974004800 0x7f79900cc0a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f798f200640 1 -- 172.21.15.40:0/620686028 >> 172.21.15.40:0/620686028 conn(0x7f799006c700 msgr2=0x7f79900c9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 -- 172.21.15.40:0/1549950302 shutdown_connections 2024-03-08T06:04:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f391806b520 0x7f391806d9e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f393c15e840 0x7f393c1f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f393c15dd50 0x7f393c0cbb80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 --2- 172.21.15.40:0/1549950302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f393c06c7d0 0x7f393c1f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 -- 172.21.15.40:0/1549950302 >> 172.21.15.40:0/1549950302 conn(0x7f393c066fc0 msgr2=0x7f393c15b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 -- 172.21.15.40:0/1549950302 shutdown_connections 2024-03-08T06:04:10.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.259+0000 7f3941e00640 1 -- 172.21.15.40:0/1549950302 wait complete. 2024-03-08T06:04:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.260+0000 7f798f200640 1 -- 172.21.15.40:0/620686028 shutdown_connections 2024-03-08T06:04:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.260+0000 7f798f200640 1 -- 172.21.15.40:0/620686028 wait complete. 2024-03-08T06:04:10.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.260+0000 7fb2fca00640 1 -- 172.21.15.40:0/1929395716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb2f8066ea0 con 0x7fb2f80cfb70 2024-03-08T06:04:10.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-03-08T06:04:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.260+0000 7fb2ee000640 1 -- 172.21.15.40:0/1929395716 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb2f8066ea0 con 0x7fb2f80cfb70 2024-03-08T06:04:10.262 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2024-03-08T06:04:10.262 DEBUG:teuthology.parallel:result is None 2024-03-08T06:04:10.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.263+0000 7fb2e3400640 1 -- 172.21.15.40:0/1929395716 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb2d806b5a0 msgr2=0x7fb2d806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.263+0000 7fb2e3400640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb2d806b5a0 0x7fb2d806da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8006fd0 tx=0x7fb2e8008040 comp rx=0 tx=0).stop 2024-03-08T06:04:10.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.263+0000 7fb2e3400640 1 -- 172.21.15.40:0/1929395716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2f80cfb70 msgr2=0x7fb2f81e5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.263+0000 7fb2e3400640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2f80cfb70 0x7fb2f81e5570 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0036040 tx=0x7fb2f0004a40 comp rx=0 tx=0).stop 2024-03-08T06:04:10.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.263+0000 7fb2e3400640 1 -- 172.21.15.40:0/1929395716 shutdown_connections 2024-03-08T06:04:10.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.263+0000 7fb2e3400640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb2d806b5a0 0x7fb2d806da60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.263+0000 7fb2e3400640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2f815be20 0x7fb2f80c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.263+0000 7fb2e3400640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2f80cfb70 0x7fb2f81e5570 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.263+0000 7fb2e3400640 1 --2- 172.21.15.40:0/1929395716 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb2f80cc480 0x7fb2f81f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.263+0000 7fb2e3400640 1 -- 172.21.15.40:0/1929395716 >> 172.21.15.40:0/1929395716 conn(0x7fb2f80b3460 msgr2=0x7fb2f80b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:10.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.264+0000 7fb2e3400640 1 -- 172.21.15.40:0/1929395716 shutdown_connections 2024-03-08T06:04:10.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.264+0000 7fb2e3400640 1 -- 172.21.15.40:0/1929395716 wait complete. 2024-03-08T06:04:10.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.264+0000 7f090c800640 1 -- 172.21.15.40:0/2020733551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f090415d2f0 con 0x7f08ec003a00 2024-03-08T06:04:10.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.265+0000 7f08fe000640 1 -- 172.21.15.40:0/2020733551 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f08f804c020 con 0x7f08ec003a00 2024-03-08T06:04:10.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.265+0000 7f94f3e00640 1 -- 172.21.15.40:0/1993186108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f94ec142240 con 0x7f94ec15efb0 2024-03-08T06:04:10.266 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-03-08T06:04:10.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.266+0000 7f94e9600640 1 -- 172.21.15.40:0/1993186108 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f94ec139010 con 0x7f94ec15efb0 2024-03-08T06:04:10.267 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-03-08T06:04:10.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f090c800640 1 -- 172.21.15.40:0/2020733551 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f08e806b5a0 msgr2=0x7f08e806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f090c800640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f08e806b5a0 0x7f08e806da60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08f40019b0 tx=0x7f08f4009040 comp rx=0 tx=0).stop 2024-03-08T06:04:10.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f090c800640 1 -- 172.21.15.40:0/2020733551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 msgr2=0x7f09041379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f090c800640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 0x7f09041379e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009be0 tx=0x7f08f8004730 comp rx=0 tx=0).stop 2024-03-08T06:04:10.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f090c800640 1 -- 172.21.15.40:0/2020733551 shutdown_connections 2024-03-08T06:04:10.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f090c800640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f08e806b5a0 0x7f08e806da60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f090c800640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f09040b0810 0x7f0904139060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f090c800640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0904156400 0x7f09040b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f090c800640 1 --2- 172.21.15.40:0/2020733551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08ec003a00 0x7f09041379e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f090c800640 1 -- 172.21.15.40:0/2020733551 >> 172.21.15.40:0/2020733551 conn(0x7f090406ac50 msgr2=0x7f0904142060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:10.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f94f3e00640 1 -- 172.21.15.40:0/1993186108 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f94cc06b4d0 msgr2=0x7f94cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f94f3e00640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f94cc06b4d0 0x7f94cc06d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f94e4036000 tx=0x7f94e4030000 comp rx=0 tx=0).stop 2024-03-08T06:04:10.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f94f3e00640 1 -- 172.21.15.40:0/1993186108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ec15efb0 msgr2=0x7f94ec146930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f94f3e00640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ec15efb0 0x7f94ec146930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94d800c780 tx=0x7f94d800cc40 comp rx=0 tx=0).stop 2024-03-08T06:04:10.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.270+0000 7f090c800640 1 -- 172.21.15.40:0/2020733551 shutdown_connections 2024-03-08T06:04:10.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.271+0000 7f94f3e00640 1 -- 172.21.15.40:0/1993186108 shutdown_connections 2024-03-08T06:04:10.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.271+0000 7f090c800640 1 -- 172.21.15.40:0/2020733551 wait complete. 2024-03-08T06:04:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.271+0000 7f94f3e00640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f94cc06b4d0 0x7f94cc06d990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.271+0000 7f94f3e00640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ec15efb0 0x7f94ec146930 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.271+0000 7f94f3e00640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ec147690 0x7f94ec1463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.271+0000 7f94f3e00640 1 --2- 172.21.15.40:0/1993186108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ec071f10 0x7f94ec15d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.271+0000 7f94f3e00640 1 -- 172.21.15.40:0/1993186108 >> 172.21.15.40:0/1993186108 conn(0x7f94ec06ac50 msgr2=0x7f94ec142740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.271+0000 7f94f3e00640 1 -- 172.21.15.40:0/1993186108 shutdown_connections 2024-03-08T06:04:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.271+0000 7f94f3e00640 1 -- 172.21.15.40:0/1993186108 wait complete. 2024-03-08T06:04:10.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-03-08T06:04:10.275 DEBUG:teuthology.parallel:result is None 2024-03-08T06:04:10.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2024-03-08T06:04:10.285 DEBUG:teuthology.parallel:result is None 2024-03-08T06:04:10.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2024-03-08T06:04:10.287 DEBUG:teuthology.parallel:result is None 2024-03-08T06:04:10.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-03-08T06:04:10.287 DEBUG:teuthology.parallel:result is None 2024-03-08T06:04:10.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2024-03-08T06:04:10.297 DEBUG:teuthology.parallel:result is None 2024-03-08T06:04:10.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2024-03-08T06:04:10.299 DEBUG:teuthology.parallel:result is None 2024-03-08T06:04:10.299 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-08T06:04:10.299 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:04:10.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.555+0000 7f8e63e00640 1 Processor -- start 2024-03-08T06:04:10.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.555+0000 7f8e63e00640 1 -- start start 2024-03-08T06:04:10.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.556+0000 7f8e63e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8e6415dce0 0x7f8e640d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:10.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.556+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8e640d3690 0x7f8e640b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:10.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e63e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e640b7ca0 con 0x7f8e6406c290 2024-03-08T06:04:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e63e00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e640b9b80 con 0x7f8e6415dce0 2024-03-08T06:04:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e63e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e640d25f0 con 0x7f8e640d3690 2024-03-08T06:04:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e62a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8e6415dce0 0x7f8e640d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e63400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8e6406c290 0x7f8e640b5600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60956/0 (socket says 172.21.15.40:60956) 2024-03-08T06:04:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e63400640 1 -- 172.21.15.40:0/573383420 learned_addr learned my addr 172.21.15.40:0/573383420 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e62000640 1 --2- 172.21.15.40:0/573383420 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8e640d3690 0x7f8e640b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e62a00640 1 -- 172.21.15.40:0/573383420 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8e640d3690 msgr2=0x7f8e640b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e62a00640 1 --2- 172.21.15.40:0/573383420 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8e640d3690 0x7f8e640b9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e62a00640 1 -- 172.21.15.40:0/573383420 >> v1:172.21.15.40:6789/0 conn(0x7f8e6406c290 legacy=0x7f8e640b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e62a00640 1 -- 172.21.15.40:0/573383420 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e640d28d0 con 0x7f8e6415dce0 2024-03-08T06:04:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.557+0000 7f8e61600640 1 -- 172.21.15.40:0/573383420 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094128831 0 0) 0x7f8e640b7ca0 con 0x7f8e6406c290 2024-03-08T06:04:10.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.559+0000 7f8e62a00640 1 --2- 172.21.15.40:0/573383420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e6415dce0 0x7f8e640d1fa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8e540097a0 tx=0x7f8e5402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea37b85e3777278e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:10.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.559+0000 7f8e61600640 1 -- 172.21.15.40:0/573383420 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54034070 con 0x7f8e6415dce0 2024-03-08T06:04:10.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.559+0000 7f8e61600640 1 -- 172.21.15.40:0/573383420 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e54030610 con 0x7f8e6415dce0 2024-03-08T06:04:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.559+0000 7f8e61600640 1 -- 172.21.15.40:0/573383420 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e540309e0 con 0x7f8e6415dce0 2024-03-08T06:04:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.560+0000 7f8e63e00640 1 -- 172.21.15.40:0/573383420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e6415dce0 msgr2=0x7f8e640d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.560+0000 7f8e63e00640 1 --2- 172.21.15.40:0/573383420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e6415dce0 0x7f8e640d1fa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8e540097a0 tx=0x7f8e5402b960 comp rx=0 tx=0).stop 2024-03-08T06:04:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.560+0000 7f8e63e00640 1 -- 172.21.15.40:0/573383420 shutdown_connections 2024-03-08T06:04:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.560+0000 7f8e63e00640 1 --2- 172.21.15.40:0/573383420 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8e640d3690 0x7f8e640b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.560+0000 7f8e63e00640 1 --2- 172.21.15.40:0/573383420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e6415dce0 0x7f8e640d1fa0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.560+0000 7f8e63e00640 1 -- 172.21.15.40:0/573383420 >> 172.21.15.40:0/573383420 conn(0x7f8e64066fc0 msgr2=0x7f8e640d3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.560+0000 7f8e63e00640 1 -- 172.21.15.40:0/573383420 shutdown_connections 2024-03-08T06:04:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.560+0000 7f8e63e00640 1 -- 172.21.15.40:0/573383420 wait complete. 2024-03-08T06:04:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.562+0000 7f8e63e00640 1 Processor -- start 2024-03-08T06:04:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.562+0000 7f8e63e00640 1 -- start start 2024-03-08T06:04:10.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.563+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e6406c290 0x7f8e640cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.563+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8e640d3690 0x7f8e64150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.563+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e6415dce0 0x7f8e640cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.563+0000 7f8e63e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e640d3170 con 0x7f8e6415dce0 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.563+0000 7f8e63e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e640d2e90 con 0x7f8e6406c290 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.563+0000 7f8e63e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6415e0c0 con 0x7f8e640d3690 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.563+0000 7f8e63400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e6415dce0 0x7f8e640cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.563+0000 7f8e63400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e6415dce0 0x7f8e640cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45782/0 (socket says 172.21.15.40:45782) 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.563+0000 7f8e63400640 1 -- 172.21.15.40:0/2216983629 learned_addr learned my addr 172.21.15.40:0/2216983629 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e62a00640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e6406c290 0x7f8e640cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e62000640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8e640d3690 0x7f8e64150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e63400640 1 -- 172.21.15.40:0/2216983629 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8e640d3690 msgr2=0x7f8e64150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e63400640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8e640d3690 0x7f8e64150000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e63400640 1 -- 172.21.15.40:0/2216983629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e6406c290 msgr2=0x7f8e640cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e63400640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e6406c290 0x7f8e640cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e63400640 1 -- 172.21.15.40:0/2216983629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6414ef60 con 0x7f8e6415dce0 2024-03-08T06:04:10.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e63400640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e6415dce0 0x7f8e640cb780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00c7d0 tx=0x7f8e5c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:10.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e5aa00640 1 -- 172.21.15.40:0/2216983629 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c018070 con 0x7f8e6415dce0 2024-03-08T06:04:10.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e5aa00640 1 -- 172.21.15.40:0/2216983629 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5c00a640 con 0x7f8e6415dce0 2024-03-08T06:04:10.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e63e00640 1 -- 172.21.15.40:0/2216983629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6414f240 con 0x7f8e6415dce0 2024-03-08T06:04:10.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e5aa00640 1 -- 172.21.15.40:0/2216983629 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c00aa70 con 0x7f8e6415dce0 2024-03-08T06:04:10.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.564+0000 7f8e63e00640 1 -- 172.21.15.40:0/2216983629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6414f800 con 0x7f8e6415dce0 2024-03-08T06:04:10.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.566+0000 7f8e63e00640 1 -- 172.21.15.40:0/2216983629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e64075100 con 0x7f8e6415dce0 2024-03-08T06:04:10.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.566+0000 7f8e5aa00640 1 -- 172.21.15.40:0/2216983629 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e5c01b070 con 0x7f8e6415dce0 2024-03-08T06:04:10.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.568+0000 7f8e5aa00640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8e4406b4d0 0x7f8e4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:10.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.568+0000 7f8e5aa00640 1 -- 172.21.15.40:0/2216983629 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e5c089820 con 0x7f8e6415dce0 2024-03-08T06:04:10.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.568+0000 7f8e62a00640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8e4406b4d0 0x7f8e4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:10.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.569+0000 7f8e62a00640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8e4406b4d0 0x7f8e4406d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8e54036000 tx=0x7f8e5403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:10.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.572+0000 7f8e5aa00640 1 -- 172.21.15.40:0/2216983629 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e5c0586a0 con 0x7f8e6415dce0 2024-03-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.759+0000 7f8e63e00640 1 -- 172.21.15.40:0/2216983629 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e640b8990 con 0x7f8e4406b4d0 2024-03-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.779+0000 7f8e5aa00640 1 -- 172.21.15.40:0/2216983629 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251874 (secure 0 0 0) 0x7f8e640b8990 con 0x7f8e4406b4d0 2024-03-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-03-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.783+0000 7f8e63e00640 1 -- 172.21.15.40:0/2216983629 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8e4406b4d0 msgr2=0x7f8e4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.783+0000 7f8e63e00640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8e4406b4d0 0x7f8e4406d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8e54036000 tx=0x7f8e5403b040 comp rx=0 tx=0).stop 2024-03-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.783+0000 7f8e63e00640 1 -- 172.21.15.40:0/2216983629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e6415dce0 msgr2=0x7f8e640cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.783+0000 7f8e63e00640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e6415dce0 0x7f8e640cb780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00c7d0 tx=0x7f8e5c00cc90 comp rx=0 tx=0).stop 2024-03-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.783+0000 7f8e63e00640 1 -- 172.21.15.40:0/2216983629 shutdown_connections 2024-03-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.783+0000 7f8e63e00640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8e4406b4d0 0x7f8e4406d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.783+0000 7f8e63e00640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e6415dce0 0x7f8e640cb780 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.783+0000 7f8e63e00640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8e640d3690 0x7f8e64150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.783+0000 7f8e63e00640 1 --2- 172.21.15.40:0/2216983629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e6406c290 0x7f8e640cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.783+0000 7f8e63e00640 1 -- 172.21.15.40:0/2216983629 >> 172.21.15.40:0/2216983629 conn(0x7f8e64066fc0 msgr2=0x7f8e6415aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.784+0000 7f8e63e00640 1 -- 172.21.15.40:0/2216983629 shutdown_connections 2024-03-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:10.784+0000 7f8e63e00640 1 -- 172.21.15.40:0/2216983629 wait complete. 2024-03-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-08T06:04:09.484838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235956,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738764,"statfs":{"total":773094113280,"available":772852494336,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.968226"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618708+0000","last_change":"2024-03-08T06:04:02.552950+0000","last_active":"2024-03-08T06:04:03.618708+0000","last_peered":"2024-03-08T06:04:03.618708+0000","last_clean":"2024-03-08T06:04:03.618708+0000","last_became_active":"2024-03-08T06:04:02.552600+0000","last_became_peered":"2024-03-08T06:04:02.552600+0000","last_unstale":"2024-03-08T06:04:03.618708+0000","last_undegraded":"2024-03-08T06:04:03.618708+0000","last_fullsized":"2024-03-08T06:04:03.618708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:13:36.393811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553892+0000","last_change":"2024-03-08T06:04:02.547475+0000","last_active":"2024-03-08T06:04:03.553892+0000","last_peered":"2024-03-08T06:04:03.553892+0000","last_clean":"2024-03-08T06:04:03.553892+0000","last_became_active":"2024-03-08T06:04:02.545953+0000","last_became_peered":"2024-03-08T06:04:02.545953+0000","last_unstale":"2024-03-08T06:04:03.553892+0000","last_undegraded":"2024-03-08T06:04:03.553892+0000","last_fullsized":"2024-03-08T06:04:03.553892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719170+0000","last_change":"2024-03-08T06:04:02.544781+0000","last_active":"2024-03-08T06:04:04.719170+0000","last_peered":"2024-03-08T06:04:04.719170+0000","last_clean":"2024-03-08T06:04:04.719170+0000","last_became_active":"2024-03-08T06:04:02.543995+0000","last_became_peered":"2024-03-08T06:04:02.543995+0000","last_unstale":"2024-03-08T06:04:04.719170+0000","last_undegraded":"2024-03-08T06:04:04.719170+0000","last_fullsized":"2024-03-08T06:04:04.719170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515851+0000","last_change":"2024-03-08T06:04:02.551644+0000","last_active":"2024-03-08T06:04:03.515851+0000","last_peered":"2024-03-08T06:04:03.515851+0000","last_clean":"2024-03-08T06:04:03.515851+0000","last_became_active":"2024-03-08T06:04:02.550930+0000","last_became_peered":"2024-03-08T06:04:02.550930+0000","last_unstale":"2024-03-08T06:04:03.515851+0000","last_undegraded":"2024-03-08T06:04:03.515851+0000","last_fullsized":"2024-03-08T06:04:03.515851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:23:55.655734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617514+0000","last_change":"2024-03-08T06:04:02.545560+0000","last_active":"2024-03-08T06:04:03.617514+0000","last_peered":"2024-03-08T06:04:03.617514+0000","last_clean":"2024-03-08T06:04:03.617514+0000","last_became_active":"2024-03-08T06:04:02.543855+0000","last_became_peered":"2024-03-08T06:04:02.543855+0000","last_unstale":"2024-03-08T06:04:03.617514+0000","last_undegraded":"2024-03-08T06:04:03.617514+0000","last_fullsized":"2024-03-08T06:04:03.617514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515916+0000","last_change":"2024-03-08T06:04:02.550172+0000","last_active":"2024-03-08T06:04:03.515916+0000","last_peered":"2024-03-08T06:04:03.515916+0000","last_clean":"2024-03-08T06:04:03.515916+0000","last_became_active":"2024-03-08T06:04:02.549807+0000","last_became_peered":"2024-03-08T06:04:02.549807+0000","last_unstale":"2024-03-08T06:04:03.515916+0000","last_undegraded":"2024-03-08T06:04:03.515916+0000","last_fullsized":"2024-03-08T06:04:03.515916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617257+0000","last_change":"2024-03-08T06:04:02.547691+0000","last_active":"2024-03-08T06:04:03.617257+0000","last_peered":"2024-03-08T06:04:03.617257+0000","last_clean":"2024-03-08T06:04:03.617257+0000","last_became_active":"2024-03-08T06:04:02.547325+0000","last_became_peered":"2024-03-08T06:04:02.547325+0000","last_unstale":"2024-03-08T06:04:03.617257+0000","last_undegraded":"2024-03-08T06:04:03.617257+0000","last_fullsized":"2024-03-08T06:04:03.617257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511377+0000","last_change":"2024-03-08T06:04:02.553110+0000","last_active":"2024-03-08T06:04:03.511377+0000","last_peered":"2024-03-08T06:04:03.511377+0000","last_clean":"2024-03-08T06:04:03.511377+0000","last_became_active":"2024-03-08T06:04:02.552635+0000","last_became_peered":"2024-03-08T06:04:02.552635+0000","last_unstale":"2024-03-08T06:04:03.511377+0000","last_undegraded":"2024-03-08T06:04:03.511377+0000","last_fullsized":"2024-03-08T06:04:03.511377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618751+0000","last_change":"2024-03-08T06:04:02.570801+0000","last_active":"2024-03-08T06:04:03.618751+0000","last_peered":"2024-03-08T06:04:03.618751+0000","last_clean":"2024-03-08T06:04:03.618751+0000","last_became_active":"2024-03-08T06:04:02.570558+0000","last_became_peered":"2024-03-08T06:04:02.570558+0000","last_unstale":"2024-03-08T06:04:03.618751+0000","last_undegraded":"2024-03-08T06:04:03.618751+0000","last_fullsized":"2024-03-08T06:04:03.618751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:07:57.853863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559891+0000","last_change":"2024-03-08T06:04:02.562132+0000","last_active":"2024-03-08T06:04:03.559891+0000","last_peered":"2024-03-08T06:04:03.559891+0000","last_clean":"2024-03-08T06:04:03.559891+0000","last_became_active":"2024-03-08T06:04:02.561851+0000","last_became_peered":"2024-03-08T06:04:02.561851+0000","last_unstale":"2024-03-08T06:04:03.559891+0000","last_undegraded":"2024-03-08T06:04:03.559891+0000","last_fullsized":"2024-03-08T06:04:03.559891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559579+0000","last_change":"2024-03-08T06:04:02.546846+0000","last_active":"2024-03-08T06:04:03.559579+0000","last_peered":"2024-03-08T06:04:03.559579+0000","last_clean":"2024-03-08T06:04:03.559579+0000","last_became_active":"2024-03-08T06:04:02.546522+0000","last_became_peered":"2024-03-08T06:04:02.546522+0000","last_unstale":"2024-03-08T06:04:03.559579+0000","last_undegraded":"2024-03-08T06:04:03.559579+0000","last_fullsized":"2024-03-08T06:04:03.559579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118422+0000","last_change":"2024-03-08T06:04:02.555942+0000","last_active":"2024-03-08T06:04:04.118422+0000","last_peered":"2024-03-08T06:04:04.118422+0000","last_clean":"2024-03-08T06:04:04.118422+0000","last_became_active":"2024-03-08T06:04:02.555651+0000","last_became_peered":"2024-03-08T06:04:02.555651+0000","last_unstale":"2024-03-08T06:04:04.118422+0000","last_undegraded":"2024-03-08T06:04:04.118422+0000","last_fullsized":"2024-03-08T06:04:04.118422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:57.032278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511828+0000","last_change":"2024-03-08T06:04:02.551969+0000","last_active":"2024-03-08T06:04:03.511828+0000","last_peered":"2024-03-08T06:04:03.511828+0000","last_clean":"2024-03-08T06:04:03.511828+0000","last_became_active":"2024-03-08T06:04:02.551664+0000","last_became_peered":"2024-03-08T06:04:02.551664+0000","last_unstale":"2024-03-08T06:04:03.511828+0000","last_undegraded":"2024-03-08T06:04:03.511828+0000","last_fullsized":"2024-03-08T06:04:03.511828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.718486+0000","last_change":"2024-03-08T06:04:02.542608+0000","last_active":"2024-03-08T06:04:04.718486+0000","last_peered":"2024-03-08T06:04:04.718486+0000","last_clean":"2024-03-08T06:04:04.718486+0000","last_became_active":"2024-03-08T06:04:02.542318+0000","last_became_peered":"2024-03-08T06:04:02.542318+0000","last_unstale":"2024-03-08T06:04:04.718486+0000","last_undegraded":"2024-03-08T06:04:04.718486+0000","last_fullsized":"2024-03-08T06:04:04.718486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:04:14.473084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553245+0000","last_change":"2024-03-08T06:04:02.548664+0000","last_active":"2024-03-08T06:04:03.553245+0000","last_peered":"2024-03-08T06:04:03.553245+0000","last_clean":"2024-03-08T06:04:03.553245+0000","last_became_active":"2024-03-08T06:04:02.547601+0000","last_became_peered":"2024-03-08T06:04:02.547601+0000","last_unstale":"2024-03-08T06:04:03.553245+0000","last_undegraded":"2024-03-08T06:04:03.553245+0000","last_fullsized":"2024-03-08T06:04:03.553245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515232+0000","last_change":"2024-03-08T06:04:02.551648+0000","last_active":"2024-03-08T06:04:03.515232+0000","last_peered":"2024-03-08T06:04:03.515232+0000","last_clean":"2024-03-08T06:04:03.515232+0000","last_became_active":"2024-03-08T06:04:02.551290+0000","last_became_peered":"2024-03-08T06:04:02.551290+0000","last_unstale":"2024-03-08T06:04:03.515232+0000","last_undegraded":"2024-03-08T06:04:03.515232+0000","last_fullsized":"2024-03-08T06:04:03.515232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.518022+0000","last_change":"2024-03-08T06:04:02.556704+0000","last_active":"2024-03-08T06:04:03.518022+0000","last_peered":"2024-03-08T06:04:03.518022+0000","last_clean":"2024-03-08T06:04:03.518022+0000","last_became_active":"2024-03-08T06:04:02.556429+0000","last_became_peered":"2024-03-08T06:04:02.556429+0000","last_unstale":"2024-03-08T06:04:03.518022+0000","last_undegraded":"2024-03-08T06:04:03.518022+0000","last_fullsized":"2024-03-08T06:04:03.518022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:13:36.393811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.697123+0000","last_change":"2024-03-08T06:04:02.539066+0000","last_active":"2024-03-08T06:04:04.697123+0000","last_peered":"2024-03-08T06:04:04.697123+0000","last_clean":"2024-03-08T06:04:04.697123+0000","last_became_active":"2024-03-08T06:04:02.538745+0000","last_became_peered":"2024-03-08T06:04:02.538745+0000","last_unstale":"2024-03-08T06:04:04.697123+0000","last_undegraded":"2024-03-08T06:04:04.697123+0000","last_fullsized":"2024-03-08T06:04:04.697123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:06:00.048492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618590+0000","last_change":"2024-03-08T06:04:02.551987+0000","last_active":"2024-03-08T06:04:03.618590+0000","last_peered":"2024-03-08T06:04:03.618590+0000","last_clean":"2024-03-08T06:04:03.618590+0000","last_became_active":"2024-03-08T06:04:02.551425+0000","last_became_peered":"2024-03-08T06:04:02.551425+0000","last_unstale":"2024-03-08T06:04:03.618590+0000","last_undegraded":"2024-03-08T06:04:03.618590+0000","last_fullsized":"2024-03-08T06:04:03.618590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:41:43.397781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559135+0000","last_change":"2024-03-08T06:04:02.559334+0000","last_active":"2024-03-08T06:04:03.559135+0000","last_peered":"2024-03-08T06:04:03.559135+0000","last_clean":"2024-03-08T06:04:03.559135+0000","last_became_active":"2024-03-08T06:04:02.558979+0000","last_became_peered":"2024-03-08T06:04:02.558979+0000","last_unstale":"2024-03-08T06:04:03.559135+0000","last_undegraded":"2024-03-08T06:04:03.559135+0000","last_fullsized":"2024-03-08T06:04:03.559135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719096+0000","last_change":"2024-03-08T06:04:02.545858+0000","last_active":"2024-03-08T06:04:04.719096+0000","last_peered":"2024-03-08T06:04:04.719096+0000","last_clean":"2024-03-08T06:04:04.719096+0000","last_became_active":"2024-03-08T06:04:02.545518+0000","last_became_peered":"2024-03-08T06:04:02.545518+0000","last_unstale":"2024-03-08T06:04:04.719096+0000","last_undegraded":"2024-03-08T06:04:04.719096+0000","last_fullsized":"2024-03-08T06:04:04.719096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516005+0000","last_change":"2024-03-08T06:04:02.553462+0000","last_active":"2024-03-08T06:04:03.516005+0000","last_peered":"2024-03-08T06:04:03.516005+0000","last_clean":"2024-03-08T06:04:03.516005+0000","last_became_active":"2024-03-08T06:04:02.552688+0000","last_became_peered":"2024-03-08T06:04:02.552688+0000","last_unstale":"2024-03-08T06:04:03.516005+0000","last_undegraded":"2024-03-08T06:04:03.516005+0000","last_fullsized":"2024-03-08T06:04:03.516005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511690+0000","last_change":"2024-03-08T06:04:02.560361+0000","last_active":"2024-03-08T06:04:03.511690+0000","last_peered":"2024-03-08T06:04:03.511690+0000","last_clean":"2024-03-08T06:04:03.511690+0000","last_became_active":"2024-03-08T06:04:02.560083+0000","last_became_peered":"2024-03-08T06:04:02.560083+0000","last_unstale":"2024-03-08T06:04:03.511690+0000","last_undegraded":"2024-03-08T06:04:03.511690+0000","last_fullsized":"2024-03-08T06:04:03.511690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515403+0000","last_change":"2024-03-08T06:04:02.549163+0000","last_active":"2024-03-08T06:04:03.515403+0000","last_peered":"2024-03-08T06:04:03.515403+0000","last_clean":"2024-03-08T06:04:03.515403+0000","last_became_active":"2024-03-08T06:04:02.548715+0000","last_became_peered":"2024-03-08T06:04:02.548715+0000","last_unstale":"2024-03-08T06:04:03.515403+0000","last_undegraded":"2024-03-08T06:04:03.515403+0000","last_fullsized":"2024-03-08T06:04:03.515403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512314+0000","last_change":"2024-03-08T06:04:02.565662+0000","last_active":"2024-03-08T06:04:03.512314+0000","last_peered":"2024-03-08T06:04:03.512314+0000","last_clean":"2024-03-08T06:04:03.512314+0000","last_became_active":"2024-03-08T06:04:02.565344+0000","last_became_peered":"2024-03-08T06:04:02.565344+0000","last_unstale":"2024-03-08T06:04:03.512314+0000","last_undegraded":"2024-03-08T06:04:03.512314+0000","last_fullsized":"2024-03-08T06:04:03.512314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:23:55.655734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.560225+0000","last_change":"2024-03-08T06:04:02.545865+0000","last_active":"2024-03-08T06:04:03.560225+0000","last_peered":"2024-03-08T06:04:03.560225+0000","last_clean":"2024-03-08T06:04:03.560225+0000","last_became_active":"2024-03-08T06:04:02.543828+0000","last_became_peered":"2024-03-08T06:04:02.543828+0000","last_unstale":"2024-03-08T06:04:03.560225+0000","last_undegraded":"2024-03-08T06:04:03.560225+0000","last_fullsized":"2024-03-08T06:04:03.560225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553769+0000","last_change":"2024-03-08T06:04:02.544859+0000","last_active":"2024-03-08T06:04:03.553769+0000","last_peered":"2024-03-08T06:04:03.553769+0000","last_clean":"2024-03-08T06:04:03.553769+0000","last_became_active":"2024-03-08T06:04:02.544502+0000","last_became_peered":"2024-03-08T06:04:02.544502+0000","last_unstale":"2024-03-08T06:04:03.553769+0000","last_undegraded":"2024-03-08T06:04:03.553769+0000","last_fullsized":"2024-03-08T06:04:03.553769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512315+0000","last_change":"2024-03-08T06:04:02.563733+0000","last_active":"2024-03-08T06:04:03.512315+0000","last_peered":"2024-03-08T06:04:03.512315+0000","last_clean":"2024-03-08T06:04:03.512315+0000","last_became_active":"2024-03-08T06:04:02.563420+0000","last_became_peered":"2024-03-08T06:04:02.563420+0000","last_unstale":"2024-03-08T06:04:03.512315+0000","last_undegraded":"2024-03-08T06:04:03.512315+0000","last_fullsized":"2024-03-08T06:04:03.512315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719077+0000","last_change":"2024-03-08T06:04:02.548150+0000","last_active":"2024-03-08T06:04:04.719077+0000","last_peered":"2024-03-08T06:04:04.719077+0000","last_clean":"2024-03-08T06:04:04.719077+0000","last_became_active":"2024-03-08T06:04:02.547846+0000","last_became_peered":"2024-03-08T06:04:02.547846+0000","last_unstale":"2024-03-08T06:04:04.719077+0000","last_undegraded":"2024-03-08T06:04:04.719077+0000","last_fullsized":"2024-03-08T06:04:04.719077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516515+0000","last_change":"2024-03-08T06:04:02.547204+0000","last_active":"2024-03-08T06:04:03.516515+0000","last_peered":"2024-03-08T06:04:03.516515+0000","last_clean":"2024-03-08T06:04:03.516515+0000","last_became_active":"2024-03-08T06:04:02.546850+0000","last_became_peered":"2024-03-08T06:04:02.546850+0000","last_unstale":"2024-03-08T06:04:03.516515+0000","last_undegraded":"2024-03-08T06:04:03.516515+0000","last_fullsized":"2024-03-08T06:04:03.516515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:32:53.947147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.696799+0000","last_change":"2024-03-08T06:04:02.553107+0000","last_active":"2024-03-08T06:04:04.696799+0000","last_peered":"2024-03-08T06:04:04.696799+0000","last_clean":"2024-03-08T06:04:04.696799+0000","last_became_active":"2024-03-08T06:04:02.552862+0000","last_became_peered":"2024-03-08T06:04:02.552862+0000","last_unstale":"2024-03-08T06:04:04.696799+0000","last_undegraded":"2024-03-08T06:04:04.696799+0000","last_fullsized":"2024-03-08T06:04:04.696799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:57.032278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559013+0000","last_change":"2024-03-08T06:04:02.544639+0000","last_active":"2024-03-08T06:04:03.559013+0000","last_peered":"2024-03-08T06:04:03.559013+0000","last_clean":"2024-03-08T06:04:03.559013+0000","last_became_active":"2024-03-08T06:04:02.544193+0000","last_became_peered":"2024-03-08T06:04:02.544193+0000","last_unstale":"2024-03-08T06:04:03.559013+0000","last_undegraded":"2024-03-08T06:04:03.559013+0000","last_fullsized":"2024-03-08T06:04:03.559013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.117892+0000","last_change":"2024-03-08T06:04:02.558328+0000","last_active":"2024-03-08T06:04:04.117892+0000","last_peered":"2024-03-08T06:04:04.117892+0000","last_clean":"2024-03-08T06:04:04.117892+0000","last_became_active":"2024-03-08T06:04:02.555015+0000","last_became_peered":"2024-03-08T06:04:02.555015+0000","last_unstale":"2024-03-08T06:04:04.117892+0000","last_undegraded":"2024-03-08T06:04:04.117892+0000","last_fullsized":"2024-03-08T06:04:04.117892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:52.535460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512099+0000","last_change":"2024-03-08T06:04:01.555471+0000","last_active":"2024-03-08T06:04:03.512099+0000","last_peered":"2024-03-08T06:04:03.512099+0000","last_clean":"2024-03-08T06:04:03.512099+0000","last_became_active":"2024-03-08T06:04:01.555171+0000","last_became_peered":"2024-03-08T06:04:01.555171+0000","last_unstale":"2024-03-08T06:04:03.512099+0000","last_undegraded":"2024-03-08T06:04:03.512099+0000","last_fullsized":"2024-03-08T06:04:03.512099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719499+0000","last_change":"2024-03-08T06:04:02.563404+0000","last_active":"2024-03-08T06:04:04.719499+0000","last_peered":"2024-03-08T06:04:04.719499+0000","last_clean":"2024-03-08T06:04:04.719499+0000","last_became_active":"2024-03-08T06:04:02.558959+0000","last_became_peered":"2024-03-08T06:04:02.558959+0000","last_unstale":"2024-03-08T06:04:04.719499+0000","last_undegraded":"2024-03-08T06:04:04.719499+0000","last_fullsized":"2024-03-08T06:04:04.719499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559117+0000","last_change":"2024-03-08T06:04:01.549647+0000","last_active":"2024-03-08T06:04:03.559117+0000","last_peered":"2024-03-08T06:04:03.559117+0000","last_clean":"2024-03-08T06:04:03.559117+0000","last_became_active":"2024-03-08T06:04:01.549345+0000","last_became_peered":"2024-03-08T06:04:01.549345+0000","last_unstale":"2024-03-08T06:04:03.559117+0000","last_undegraded":"2024-03-08T06:04:03.559117+0000","last_fullsized":"2024-03-08T06:04:03.559117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511841+0000","last_change":"2024-03-08T06:04:01.551862+0000","last_active":"2024-03-08T06:04:03.511841+0000","last_peered":"2024-03-08T06:04:03.511841+0000","last_clean":"2024-03-08T06:04:03.511841+0000","last_became_active":"2024-03-08T06:04:01.551606+0000","last_became_peered":"2024-03-08T06:04:01.551606+0000","last_unstale":"2024-03-08T06:04:03.511841+0000","last_undegraded":"2024-03-08T06:04:03.511841+0000","last_fullsized":"2024-03-08T06:04:03.511841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:52:36.740849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512342+0000","last_change":"2024-03-08T06:04:02.548972+0000","last_active":"2024-03-08T06:04:03.512342+0000","last_peered":"2024-03-08T06:04:03.512342+0000","last_clean":"2024-03-08T06:04:03.512342+0000","last_became_active":"2024-03-08T06:04:02.548136+0000","last_became_peered":"2024-03-08T06:04:02.548136+0000","last_unstale":"2024-03-08T06:04:03.512342+0000","last_undegraded":"2024-03-08T06:04:03.512342+0000","last_fullsized":"2024-03-08T06:04:03.512342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516175+0000","last_change":"2024-03-08T06:04:01.556755+0000","last_active":"2024-03-08T06:04:03.516175+0000","last_peered":"2024-03-08T06:04:03.516175+0000","last_clean":"2024-03-08T06:04:03.516175+0000","last_became_active":"2024-03-08T06:04:01.555842+0000","last_became_peered":"2024-03-08T06:04:01.555842+0000","last_unstale":"2024-03-08T06:04:03.516175+0000","last_undegraded":"2024-03-08T06:04:03.516175+0000","last_fullsized":"2024-03-08T06:04:03.516175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553244+0000","last_change":"2024-03-08T06:04:02.549137+0000","last_active":"2024-03-08T06:04:03.553244+0000","last_peered":"2024-03-08T06:04:03.553244+0000","last_clean":"2024-03-08T06:04:03.553244+0000","last_became_active":"2024-03-08T06:04:02.547990+0000","last_became_peered":"2024-03-08T06:04:02.547990+0000","last_unstale":"2024-03-08T06:04:03.553244+0000","last_undegraded":"2024-03-08T06:04:03.553244+0000","last_fullsized":"2024-03-08T06:04:03.553244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559731+0000","last_change":"2024-03-08T06:04:01.555948+0000","last_active":"2024-03-08T06:04:03.559731+0000","last_peered":"2024-03-08T06:04:03.559731+0000","last_clean":"2024-03-08T06:04:03.559731+0000","last_became_active":"2024-03-08T06:04:01.555666+0000","last_became_peered":"2024-03-08T06:04:01.555666+0000","last_unstale":"2024-03-08T06:04:03.559731+0000","last_undegraded":"2024-03-08T06:04:03.559731+0000","last_fullsized":"2024-03-08T06:04:03.559731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517528+0000","last_change":"2024-03-08T06:04:02.558089+0000","last_active":"2024-03-08T06:04:03.517528+0000","last_peered":"2024-03-08T06:04:03.517528+0000","last_clean":"2024-03-08T06:04:03.517528+0000","last_became_active":"2024-03-08T06:04:02.557697+0000","last_became_peered":"2024-03-08T06:04:02.557697+0000","last_unstale":"2024-03-08T06:04:03.517528+0000","last_undegraded":"2024-03-08T06:04:03.517528+0000","last_fullsized":"2024-03-08T06:04:03.517528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:19:21.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517312+0000","last_change":"2024-03-08T06:04:01.559428+0000","last_active":"2024-03-08T06:04:03.517312+0000","last_peered":"2024-03-08T06:04:03.517312+0000","last_clean":"2024-03-08T06:04:03.517312+0000","last_became_active":"2024-03-08T06:04:01.559156+0000","last_became_peered":"2024-03-08T06:04:01.559156+0000","last_unstale":"2024-03-08T06:04:03.517312+0000","last_undegraded":"2024-03-08T06:04:03.517312+0000","last_fullsized":"2024-03-08T06:04:03.517312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719809+0000","last_change":"2024-03-08T06:04:02.551412+0000","last_active":"2024-03-08T06:04:04.719809+0000","last_peered":"2024-03-08T06:04:04.719809+0000","last_clean":"2024-03-08T06:04:04.719809+0000","last_became_active":"2024-03-08T06:04:02.550080+0000","last_became_peered":"2024-03-08T06:04:02.550080+0000","last_unstale":"2024-03-08T06:04:04.719809+0000","last_undegraded":"2024-03-08T06:04:04.719809+0000","last_fullsized":"2024-03-08T06:04:04.719809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553697+0000","last_change":"2024-03-08T06:04:01.554654+0000","last_active":"2024-03-08T06:04:03.553697+0000","last_peered":"2024-03-08T06:04:03.553697+0000","last_clean":"2024-03-08T06:04:03.553697+0000","last_became_active":"2024-03-08T06:04:01.554238+0000","last_became_peered":"2024-03-08T06:04:01.554238+0000","last_unstale":"2024-03-08T06:04:03.553697+0000","last_undegraded":"2024-03-08T06:04:03.553697+0000","last_fullsized":"2024-03-08T06:04:03.553697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118544+0000","last_change":"2024-03-08T06:04:02.556415+0000","last_active":"2024-03-08T06:04:04.118544+0000","last_peered":"2024-03-08T06:04:04.118544+0000","last_clean":"2024-03-08T06:04:04.118544+0000","last_became_active":"2024-03-08T06:04:02.556055+0000","last_became_peered":"2024-03-08T06:04:02.556055+0000","last_unstale":"2024-03-08T06:04:04.118544+0000","last_undegraded":"2024-03-08T06:04:04.118544+0000","last_fullsized":"2024-03-08T06:04:04.118544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:06:00.048492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559371+0000","last_change":"2024-03-08T06:04:01.549872+0000","last_active":"2024-03-08T06:04:03.559371+0000","last_peered":"2024-03-08T06:04:03.559371+0000","last_clean":"2024-03-08T06:04:03.559371+0000","last_became_active":"2024-03-08T06:04:01.549313+0000","last_became_peered":"2024-03-08T06:04:01.549313+0000","last_unstale":"2024-03-08T06:04:03.559371+0000","last_undegraded":"2024-03-08T06:04:03.559371+0000","last_fullsized":"2024-03-08T06:04:03.559371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618023+0000","last_change":"2024-03-08T06:04:02.549539+0000","last_active":"2024-03-08T06:04:03.618023+0000","last_peered":"2024-03-08T06:04:03.618023+0000","last_clean":"2024-03-08T06:04:03.618023+0000","last_became_active":"2024-03-08T06:04:02.549245+0000","last_became_peered":"2024-03-08T06:04:02.549245+0000","last_unstale":"2024-03-08T06:04:03.618023+0000","last_undegraded":"2024-03-08T06:04:03.618023+0000","last_fullsized":"2024-03-08T06:04:03.618023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:33:11.525956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719392+0000","last_change":"2024-03-08T06:04:01.542685+0000","last_active":"2024-03-08T06:04:04.719392+0000","last_peered":"2024-03-08T06:04:04.719392+0000","last_clean":"2024-03-08T06:04:04.719392+0000","last_became_active":"2024-03-08T06:04:01.542236+0000","last_became_peered":"2024-03-08T06:04:01.542236+0000","last_unstale":"2024-03-08T06:04:04.719392+0000","last_undegraded":"2024-03-08T06:04:04.719392+0000","last_fullsized":"2024-03-08T06:04:04.719392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:35:24.125408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.718718+0000","last_change":"2024-03-08T06:04:02.553210+0000","last_active":"2024-03-08T06:04:04.718718+0000","last_peered":"2024-03-08T06:04:04.718718+0000","last_clean":"2024-03-08T06:04:04.718718+0000","last_became_active":"2024-03-08T06:04:02.552889+0000","last_became_peered":"2024-03-08T06:04:02.552889+0000","last_unstale":"2024-03-08T06:04:04.718718+0000","last_undegraded":"2024-03-08T06:04:04.718718+0000","last_fullsized":"2024-03-08T06:04:04.718718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:28:17.039893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516402+0000","last_change":"2024-03-08T06:04:01.554506+0000","last_active":"2024-03-08T06:04:03.516402+0000","last_peered":"2024-03-08T06:04:03.516402+0000","last_clean":"2024-03-08T06:04:03.516402+0000","last_became_active":"2024-03-08T06:04:01.554224+0000","last_became_peered":"2024-03-08T06:04:01.554224+0000","last_unstale":"2024-03-08T06:04:03.516402+0000","last_undegraded":"2024-03-08T06:04:03.516402+0000","last_fullsized":"2024-03-08T06:04:03.516402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617833+0000","last_change":"2024-03-08T06:04:02.560754+0000","last_active":"2024-03-08T06:04:03.617833+0000","last_peered":"2024-03-08T06:04:03.617833+0000","last_clean":"2024-03-08T06:04:03.617833+0000","last_became_active":"2024-03-08T06:04:02.560482+0000","last_became_peered":"2024-03-08T06:04:02.560482+0000","last_unstale":"2024-03-08T06:04:03.617833+0000","last_undegraded":"2024-03-08T06:04:03.617833+0000","last_fullsized":"2024-03-08T06:04:03.617833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:32:53.947147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559130+0000","last_change":"2024-03-08T06:04:01.545961+0000","last_active":"2024-03-08T06:04:03.559130+0000","last_peered":"2024-03-08T06:04:03.559130+0000","last_clean":"2024-03-08T06:04:03.559130+0000","last_became_active":"2024-03-08T06:04:01.545579+0000","last_became_peered":"2024-03-08T06:04:01.545579+0000","last_unstale":"2024-03-08T06:04:03.559130+0000","last_undegraded":"2024-03-08T06:04:03.559130+0000","last_fullsized":"2024-03-08T06:04:03.559130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.558973+0000","last_change":"2024-03-08T06:04:02.544735+0000","last_active":"2024-03-08T06:04:03.558973+0000","last_peered":"2024-03-08T06:04:03.558973+0000","last_clean":"2024-03-08T06:04:03.558973+0000","last_became_active":"2024-03-08T06:04:02.544438+0000","last_became_peered":"2024-03-08T06:04:02.544438+0000","last_unstale":"2024-03-08T06:04:03.558973+0000","last_undegraded":"2024-03-08T06:04:03.558973+0000","last_fullsized":"2024-03-08T06:04:03.558973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.696817+0000","last_change":"2024-03-08T06:04:01.541060+0000","last_active":"2024-03-08T06:04:04.696817+0000","last_peered":"2024-03-08T06:04:04.696817+0000","last_clean":"2024-03-08T06:04:04.696817+0000","last_became_active":"2024-03-08T06:04:01.540657+0000","last_became_peered":"2024-03-08T06:04:01.540657+0000","last_unstale":"2024-03-08T06:04:04.696817+0000","last_undegraded":"2024-03-08T06:04:04.696817+0000","last_fullsized":"2024-03-08T06:04:04.696817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617381+0000","last_change":"2024-03-08T06:04:02.565226+0000","last_active":"2024-03-08T06:04:03.617381+0000","last_peered":"2024-03-08T06:04:03.617381+0000","last_clean":"2024-03-08T06:04:03.617381+0000","last_became_active":"2024-03-08T06:04:02.564692+0000","last_became_peered":"2024-03-08T06:04:02.564692+0000","last_unstale":"2024-03-08T06:04:03.617381+0000","last_undegraded":"2024-03-08T06:04:03.617381+0000","last_fullsized":"2024-03-08T06:04:03.617381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118978+0000","last_change":"2024-03-08T06:04:01.540655+0000","last_active":"2024-03-08T06:04:04.118978+0000","last_peered":"2024-03-08T06:04:04.118978+0000","last_clean":"2024-03-08T06:04:04.118978+0000","last_became_active":"2024-03-08T06:04:01.540205+0000","last_became_peered":"2024-03-08T06:04:01.540205+0000","last_unstale":"2024-03-08T06:04:04.118978+0000","last_undegraded":"2024-03-08T06:04:04.118978+0000","last_fullsized":"2024-03-08T06:04:04.118978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:12:32.122620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618419+0000","last_change":"2024-03-08T06:04:02.559001+0000","last_active":"2024-03-08T06:04:03.618419+0000","last_peered":"2024-03-08T06:04:03.618419+0000","last_clean":"2024-03-08T06:04:03.618419+0000","last_became_active":"2024-03-08T06:04:02.558681+0000","last_became_peered":"2024-03-08T06:04:02.558681+0000","last_unstale":"2024-03-08T06:04:03.618419+0000","last_undegraded":"2024-03-08T06:04:03.618419+0000","last_fullsized":"2024-03-08T06:04:03.618419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:19:21.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.696931+0000","last_change":"2024-03-08T06:04:01.542669+0000","last_active":"2024-03-08T06:04:04.696931+0000","last_peered":"2024-03-08T06:04:04.696931+0000","last_clean":"2024-03-08T06:04:04.696931+0000","last_became_active":"2024-03-08T06:04:01.540916+0000","last_became_peered":"2024-03-08T06:04:01.540916+0000","last_unstale":"2024-03-08T06:04:04.696931+0000","last_undegraded":"2024-03-08T06:04:04.696931+0000","last_fullsized":"2024-03-08T06:04:04.696931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618960+0000","last_change":"2024-03-08T06:04:02.569733+0000","last_active":"2024-03-08T06:04:03.618960+0000","last_peered":"2024-03-08T06:04:03.618960+0000","last_clean":"2024-03-08T06:04:03.618960+0000","last_became_active":"2024-03-08T06:04:02.567299+0000","last_became_peered":"2024-03-08T06:04:02.567299+0000","last_unstale":"2024-03-08T06:04:03.618960+0000","last_undegraded":"2024-03-08T06:04:03.618960+0000","last_fullsized":"2024-03-08T06:04:03.618960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:16:15.668237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553839+0000","last_change":"2024-03-08T06:04:01.544281+0000","last_active":"2024-03-08T06:04:03.553839+0000","last_peered":"2024-03-08T06:04:03.553839+0000","last_clean":"2024-03-08T06:04:03.553839+0000","last_became_active":"2024-03-08T06:04:01.543982+0000","last_became_peered":"2024-03-08T06:04:01.543982+0000","last_unstale":"2024-03-08T06:04:03.553839+0000","last_undegraded":"2024-03-08T06:04:03.553839+0000","last_fullsized":"2024-03-08T06:04:03.553839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:50.471516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516800+0000","last_change":"2024-03-08T06:04:02.551888+0000","last_active":"2024-03-08T06:04:03.516800+0000","last_peered":"2024-03-08T06:04:03.516800+0000","last_clean":"2024-03-08T06:04:03.516800+0000","last_became_active":"2024-03-08T06:04:02.551321+0000","last_became_peered":"2024-03-08T06:04:02.551321+0000","last_unstale":"2024-03-08T06:04:03.516800+0000","last_undegraded":"2024-03-08T06:04:03.516800+0000","last_fullsized":"2024-03-08T06:04:03.516800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:33:11.525956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.696683+0000","last_change":"2024-03-08T06:04:01.554656+0000","last_active":"2024-03-08T06:04:04.696683+0000","last_peered":"2024-03-08T06:04:04.696683+0000","last_clean":"2024-03-08T06:04:04.696683+0000","last_became_active":"2024-03-08T06:04:01.554326+0000","last_became_peered":"2024-03-08T06:04:01.554326+0000","last_unstale":"2024-03-08T06:04:04.696683+0000","last_undegraded":"2024-03-08T06:04:04.696683+0000","last_fullsized":"2024-03-08T06:04:04.696683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618096+0000","last_change":"2024-03-08T06:04:02.568164+0000","last_active":"2024-03-08T06:04:03.618096+0000","last_peered":"2024-03-08T06:04:03.618096+0000","last_clean":"2024-03-08T06:04:03.618096+0000","last_became_active":"2024-03-08T06:04:02.567123+0000","last_became_peered":"2024-03-08T06:04:02.567123+0000","last_unstale":"2024-03-08T06:04:03.618096+0000","last_undegraded":"2024-03-08T06:04:03.618096+0000","last_fullsized":"2024-03-08T06:04:03.618096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:42:12.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617235+0000","last_change":"2024-03-08T06:03:57.473597+0000","last_active":"2024-03-08T06:04:03.617235+0000","last_peered":"2024-03-08T06:04:03.617235+0000","last_clean":"2024-03-08T06:04:03.617235+0000","last_became_active":"2024-03-08T06:03:57.468593+0000","last_became_peered":"2024-03-08T06:03:57.468593+0000","last_unstale":"2024-03-08T06:04:03.617235+0000","last_undegraded":"2024-03-08T06:04:03.617235+0000","last_fullsized":"2024-03-08T06:04:03.617235+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_clean_scrub_stamp":"2024-03-08T06:03:56.214441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:45:28.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515695+0000","last_change":"2024-03-08T06:04:01.554640+0000","last_active":"2024-03-08T06:04:03.515695+0000","last_peered":"2024-03-08T06:04:03.515695+0000","last_clean":"2024-03-08T06:04:03.515695+0000","last_became_active":"2024-03-08T06:04:01.554274+0000","last_became_peered":"2024-03-08T06:04:01.554274+0000","last_unstale":"2024-03-08T06:04:03.515695+0000","last_undegraded":"2024-03-08T06:04:03.515695+0000","last_fullsized":"2024-03-08T06:04:03.515695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511891+0000","last_change":"2024-03-08T06:04:02.556830+0000","last_active":"2024-03-08T06:04:03.511891+0000","last_peered":"2024-03-08T06:04:03.511891+0000","last_clean":"2024-03-08T06:04:03.511891+0000","last_became_active":"2024-03-08T06:04:02.556071+0000","last_became_peered":"2024-03-08T06:04:02.556071+0000","last_unstale":"2024-03-08T06:04:03.511891+0000","last_undegraded":"2024-03-08T06:04:03.511891+0000","last_fullsized":"2024-03-08T06:04:03.511891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617015+0000","last_change":"2024-03-08T06:04:01.556875+0000","last_active":"2024-03-08T06:04:03.617015+0000","last_peered":"2024-03-08T06:04:03.617015+0000","last_clean":"2024-03-08T06:04:03.617015+0000","last_became_active":"2024-03-08T06:04:01.555346+0000","last_became_peered":"2024-03-08T06:04:01.555346+0000","last_unstale":"2024-03-08T06:04:03.617015+0000","last_undegraded":"2024-03-08T06:04:03.617015+0000","last_fullsized":"2024-03-08T06:04:03.617015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.718434+0000","last_change":"2024-03-08T06:04:02.563977+0000","last_active":"2024-03-08T06:04:04.718434+0000","last_peered":"2024-03-08T06:04:04.718434+0000","last_clean":"2024-03-08T06:04:04.718434+0000","last_became_active":"2024-03-08T06:04:02.563653+0000","last_became_peered":"2024-03-08T06:04:02.563653+0000","last_unstale":"2024-03-08T06:04:04.718434+0000","last_undegraded":"2024-03-08T06:04:04.718434+0000","last_fullsized":"2024-03-08T06:04:04.718434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:47:04.734023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.697051+0000","last_change":"2024-03-08T06:04:01.542592+0000","last_active":"2024-03-08T06:04:04.697051+0000","last_peered":"2024-03-08T06:04:04.697051+0000","last_clean":"2024-03-08T06:04:04.697051+0000","last_became_active":"2024-03-08T06:04:01.541567+0000","last_became_peered":"2024-03-08T06:04:01.541567+0000","last_unstale":"2024-03-08T06:04:04.697051+0000","last_undegraded":"2024-03-08T06:04:04.697051+0000","last_fullsized":"2024-03-08T06:04:04.697051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553855+0000","last_change":"2024-03-08T06:04:02.525242+0000","last_active":"2024-03-08T06:04:03.553855+0000","last_peered":"2024-03-08T06:04:03.553855+0000","last_clean":"2024-03-08T06:04:03.553855+0000","last_became_active":"2024-03-08T06:04:02.524598+0000","last_became_peered":"2024-03-08T06:04:02.524598+0000","last_unstale":"2024-03-08T06:04:03.553855+0000","last_undegraded":"2024-03-08T06:04:03.553855+0000","last_fullsized":"2024-03-08T06:04:03.553855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516807+0000","last_change":"2024-03-08T06:04:01.557363+0000","last_active":"2024-03-08T06:04:03.516807+0000","last_peered":"2024-03-08T06:04:03.516807+0000","last_clean":"2024-03-08T06:04:03.516807+0000","last_became_active":"2024-03-08T06:04:01.556516+0000","last_became_peered":"2024-03-08T06:04:01.556516+0000","last_unstale":"2024-03-08T06:04:03.516807+0000","last_undegraded":"2024-03-08T06:04:03.516807+0000","last_fullsized":"2024-03-08T06:04:03.516807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118240+0000","last_change":"2024-03-08T06:04:02.554738+0000","last_active":"2024-03-08T06:04:04.118240+0000","last_peered":"2024-03-08T06:04:04.118240+0000","last_clean":"2024-03-08T06:04:04.118240+0000","last_became_active":"2024-03-08T06:04:02.554446+0000","last_became_peered":"2024-03-08T06:04:02.554446+0000","last_unstale":"2024-03-08T06:04:04.118240+0000","last_undegraded":"2024-03-08T06:04:04.118240+0000","last_fullsized":"2024-03-08T06:04:04.118240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:51.718653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553561+0000","last_change":"2024-03-08T06:04:01.543134+0000","last_active":"2024-03-08T06:04:03.553561+0000","last_peered":"2024-03-08T06:04:03.553561+0000","last_clean":"2024-03-08T06:04:03.553561+0000","last_became_active":"2024-03-08T06:04:01.542768+0000","last_became_peered":"2024-03-08T06:04:01.542768+0000","last_unstale":"2024-03-08T06:04:03.553561+0000","last_undegraded":"2024-03-08T06:04:03.553561+0000","last_fullsized":"2024-03-08T06:04:03.553561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517531+0000","last_change":"2024-03-08T06:04:02.549139+0000","last_active":"2024-03-08T06:04:03.517531+0000","last_peered":"2024-03-08T06:04:03.517531+0000","last_clean":"2024-03-08T06:04:03.517531+0000","last_became_active":"2024-03-08T06:04:02.548765+0000","last_became_peered":"2024-03-08T06:04:02.548765+0000","last_unstale":"2024-03-08T06:04:03.517531+0000","last_undegraded":"2024-03-08T06:04:03.517531+0000","last_fullsized":"2024-03-08T06:04:03.517531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:41:43.397781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618664+0000","last_change":"2024-03-08T06:04:01.553438+0000","last_active":"2024-03-08T06:04:03.618664+0000","last_peered":"2024-03-08T06:04:03.618664+0000","last_clean":"2024-03-08T06:04:03.618664+0000","last_became_active":"2024-03-08T06:04:01.553120+0000","last_became_peered":"2024-03-08T06:04:01.553120+0000","last_unstale":"2024-03-08T06:04:03.618664+0000","last_undegraded":"2024-03-08T06:04:03.618664+0000","last_fullsized":"2024-03-08T06:04:03.618664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:05:58.801355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553556+0000","last_change":"2024-03-08T06:04:02.523565+0000","last_active":"2024-03-08T06:04:03.553556+0000","last_peered":"2024-03-08T06:04:03.553556+0000","last_clean":"2024-03-08T06:04:03.553556+0000","last_became_active":"2024-03-08T06:04:02.522603+0000","last_became_peered":"2024-03-08T06:04:02.522603+0000","last_unstale":"2024-03-08T06:04:03.553556+0000","last_undegraded":"2024-03-08T06:04:03.553556+0000","last_fullsized":"2024-03-08T06:04:03.553556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.557358+0000","last_change":"2024-03-08T06:04:01.559721+0000","last_active":"2024-03-08T06:04:03.557358+0000","last_peered":"2024-03-08T06:04:03.557358+0000","last_clean":"2024-03-08T06:04:03.557358+0000","last_became_active":"2024-03-08T06:04:01.559454+0000","last_became_peered":"2024-03-08T06:04:01.559454+0000","last_unstale":"2024-03-08T06:04:03.557358+0000","last_undegraded":"2024-03-08T06:04:03.557358+0000","last_fullsized":"2024-03-08T06:04:03.557358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512336+0000","last_change":"2024-03-08T06:04:02.555474+0000","last_active":"2024-03-08T06:04:03.512336+0000","last_peered":"2024-03-08T06:04:03.512336+0000","last_clean":"2024-03-08T06:04:03.512336+0000","last_became_active":"2024-03-08T06:04:02.551802+0000","last_became_peered":"2024-03-08T06:04:02.551802+0000","last_unstale":"2024-03-08T06:04:03.512336+0000","last_undegraded":"2024-03-08T06:04:03.512336+0000","last_fullsized":"2024-03-08T06:04:03.512336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559860+0000","last_change":"2024-03-08T06:04:01.559130+0000","last_active":"2024-03-08T06:04:03.559860+0000","last_peered":"2024-03-08T06:04:03.559860+0000","last_clean":"2024-03-08T06:04:03.559860+0000","last_became_active":"2024-03-08T06:04:01.558748+0000","last_became_peered":"2024-03-08T06:04:01.558748+0000","last_unstale":"2024-03-08T06:04:03.559860+0000","last_undegraded":"2024-03-08T06:04:03.559860+0000","last_fullsized":"2024-03-08T06:04:03.559860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719468+0000","last_change":"2024-03-08T06:04:02.544456+0000","last_active":"2024-03-08T06:04:04.719468+0000","last_peered":"2024-03-08T06:04:04.719468+0000","last_clean":"2024-03-08T06:04:04.719468+0000","last_became_active":"2024-03-08T06:04:02.543887+0000","last_became_peered":"2024-03-08T06:04:02.543887+0000","last_unstale":"2024-03-08T06:04:04.719468+0000","last_undegraded":"2024-03-08T06:04:04.719468+0000","last_fullsized":"2024-03-08T06:04:04.719468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617019+0000","last_change":"2024-03-08T06:04:01.552997+0000","last_active":"2024-03-08T06:04:03.617019+0000","last_peered":"2024-03-08T06:04:03.617019+0000","last_clean":"2024-03-08T06:04:03.617019+0000","last_became_active":"2024-03-08T06:04:01.552617+0000","last_became_peered":"2024-03-08T06:04:01.552617+0000","last_unstale":"2024-03-08T06:04:03.617019+0000","last_undegraded":"2024-03-08T06:04:03.617019+0000","last_fullsized":"2024-03-08T06:04:03.617019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118081+0000","last_change":"2024-03-08T06:04:02.554162+0000","last_active":"2024-03-08T06:04:04.118081+0000","last_peered":"2024-03-08T06:04:04.118081+0000","last_clean":"2024-03-08T06:04:04.118081+0000","last_became_active":"2024-03-08T06:04:02.553850+0000","last_became_peered":"2024-03-08T06:04:02.553850+0000","last_unstale":"2024-03-08T06:04:04.118081+0000","last_undegraded":"2024-03-08T06:04:04.118081+0000","last_fullsized":"2024-03-08T06:04:04.118081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:09.964322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553380+0000","last_change":"2024-03-08T06:04:01.547872+0000","last_active":"2024-03-08T06:04:03.553380+0000","last_peered":"2024-03-08T06:04:03.553380+0000","last_clean":"2024-03-08T06:04:03.553380+0000","last_became_active":"2024-03-08T06:04:01.547553+0000","last_became_peered":"2024-03-08T06:04:01.547553+0000","last_unstale":"2024-03-08T06:04:03.553380+0000","last_undegraded":"2024-03-08T06:04:03.553380+0000","last_fullsized":"2024-03-08T06:04:03.553380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511556+0000","last_change":"2024-03-08T06:04:02.546841+0000","last_active":"2024-03-08T06:04:03.511556+0000","last_peered":"2024-03-08T06:04:03.511556+0000","last_clean":"2024-03-08T06:04:03.511556+0000","last_became_active":"2024-03-08T06:04:02.543164+0000","last_became_peered":"2024-03-08T06:04:02.543164+0000","last_unstale":"2024-03-08T06:04:03.511556+0000","last_undegraded":"2024-03-08T06:04:03.511556+0000","last_fullsized":"2024-03-08T06:04:03.511556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553503+0000","last_change":"2024-03-08T06:04:01.543186+0000","last_active":"2024-03-08T06:04:03.553503+0000","last_peered":"2024-03-08T06:04:03.553503+0000","last_clean":"2024-03-08T06:04:03.553503+0000","last_became_active":"2024-03-08T06:04:01.542185+0000","last_became_peered":"2024-03-08T06:04:01.542185+0000","last_unstale":"2024-03-08T06:04:03.553503+0000","last_undegraded":"2024-03-08T06:04:03.553503+0000","last_fullsized":"2024-03-08T06:04:03.553503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511983+0000","last_change":"2024-03-08T06:04:02.552865+0000","last_active":"2024-03-08T06:04:03.511983+0000","last_peered":"2024-03-08T06:04:03.511983+0000","last_clean":"2024-03-08T06:04:03.511983+0000","last_became_active":"2024-03-08T06:04:02.552563+0000","last_became_peered":"2024-03-08T06:04:02.552563+0000","last_unstale":"2024-03-08T06:04:03.511983+0000","last_undegraded":"2024-03-08T06:04:03.511983+0000","last_fullsized":"2024-03-08T06:04:03.511983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618134+0000","last_change":"2024-03-08T06:04:01.560452+0000","last_active":"2024-03-08T06:04:03.618134+0000","last_peered":"2024-03-08T06:04:03.618134+0000","last_clean":"2024-03-08T06:04:03.618134+0000","last_became_active":"2024-03-08T06:04:01.559883+0000","last_became_peered":"2024-03-08T06:04:01.559883+0000","last_unstale":"2024-03-08T06:04:03.618134+0000","last_undegraded":"2024-03-08T06:04:03.618134+0000","last_fullsized":"2024-03-08T06:04:03.618134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:50.471516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559002+0000","last_change":"2024-03-08T06:04:02.562767+0000","last_active":"2024-03-08T06:04:03.559002+0000","last_peered":"2024-03-08T06:04:03.559002+0000","last_clean":"2024-03-08T06:04:03.559002+0000","last_became_active":"2024-03-08T06:04:02.562425+0000","last_became_peered":"2024-03-08T06:04:02.562425+0000","last_unstale":"2024-03-08T06:04:03.559002+0000","last_undegraded":"2024-03-08T06:04:03.559002+0000","last_fullsized":"2024-03-08T06:04:03.559002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517395+0000","last_change":"2024-03-08T06:04:02.559515+0000","last_active":"2024-03-08T06:04:03.517395+0000","last_peered":"2024-03-08T06:04:03.517395+0000","last_clean":"2024-03-08T06:04:03.517395+0000","last_became_active":"2024-03-08T06:04:02.559082+0000","last_became_peered":"2024-03-08T06:04:02.559082+0000","last_unstale":"2024-03-08T06:04:03.517395+0000","last_undegraded":"2024-03-08T06:04:03.517395+0000","last_fullsized":"2024-03-08T06:04:03.517395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:42:12.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617836+0000","last_change":"2024-03-08T06:04:01.560587+0000","last_active":"2024-03-08T06:04:03.617836+0000","last_peered":"2024-03-08T06:04:03.617836+0000","last_clean":"2024-03-08T06:04:03.617836+0000","last_became_active":"2024-03-08T06:04:01.560275+0000","last_became_peered":"2024-03-08T06:04:01.560275+0000","last_unstale":"2024-03-08T06:04:03.617836+0000","last_undegraded":"2024-03-08T06:04:03.617836+0000","last_fullsized":"2024-03-08T06:04:03.617836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516385+0000","last_change":"2024-03-08T06:04:02.559339+0000","last_active":"2024-03-08T06:04:03.516385+0000","last_peered":"2024-03-08T06:04:03.516385+0000","last_clean":"2024-03-08T06:04:03.516385+0000","last_became_active":"2024-03-08T06:04:02.559023+0000","last_became_peered":"2024-03-08T06:04:02.559023+0000","last_unstale":"2024-03-08T06:04:03.516385+0000","last_undegraded":"2024-03-08T06:04:03.516385+0000","last_fullsized":"2024-03-08T06:04:03.516385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618371+0000","last_change":"2024-03-08T06:04:01.560256+0000","last_active":"2024-03-08T06:04:03.618371+0000","last_peered":"2024-03-08T06:04:03.618371+0000","last_clean":"2024-03-08T06:04:03.618371+0000","last_became_active":"2024-03-08T06:04:01.559917+0000","last_became_peered":"2024-03-08T06:04:01.559917+0000","last_unstale":"2024-03-08T06:04:03.618371+0000","last_undegraded":"2024-03-08T06:04:03.618371+0000","last_fullsized":"2024-03-08T06:04:03.618371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:55.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719619+0000","last_change":"2024-03-08T06:04:02.551405+0000","last_active":"2024-03-08T06:04:04.719619+0000","last_peered":"2024-03-08T06:04:04.719619+0000","last_clean":"2024-03-08T06:04:04.719619+0000","last_became_active":"2024-03-08T06:04:02.550283+0000","last_became_peered":"2024-03-08T06:04:02.550283+0000","last_unstale":"2024-03-08T06:04:04.719619+0000","last_undegraded":"2024-03-08T06:04:04.719619+0000","last_fullsized":"2024-03-08T06:04:04.719619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512142+0000","last_change":"2024-03-08T06:04:01.540725+0000","last_active":"2024-03-08T06:04:03.512142+0000","last_peered":"2024-03-08T06:04:03.512142+0000","last_clean":"2024-03-08T06:04:03.512142+0000","last_became_active":"2024-03-08T06:04:01.540349+0000","last_became_peered":"2024-03-08T06:04:01.540349+0000","last_unstale":"2024-03-08T06:04:03.512142+0000","last_undegraded":"2024-03-08T06:04:03.512142+0000","last_fullsized":"2024-03-08T06:04:03.512142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512201+0000","last_change":"2024-03-08T06:04:02.554282+0000","last_active":"2024-03-08T06:04:03.512201+0000","last_peered":"2024-03-08T06:04:03.512201+0000","last_clean":"2024-03-08T06:04:03.512201+0000","last_became_active":"2024-03-08T06:04:02.551101+0000","last_became_peered":"2024-03-08T06:04:02.551101+0000","last_unstale":"2024-03-08T06:04:03.512201+0000","last_undegraded":"2024-03-08T06:04:03.512201+0000","last_fullsized":"2024-03-08T06:04:03.512201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719812+0000","last_change":"2024-03-08T06:04:01.555646+0000","last_active":"2024-03-08T06:04:04.719812+0000","last_peered":"2024-03-08T06:04:04.719812+0000","last_clean":"2024-03-08T06:04:04.719812+0000","last_became_active":"2024-03-08T06:04:01.555404+0000","last_became_peered":"2024-03-08T06:04:01.555404+0000","last_unstale":"2024-03-08T06:04:04.719812+0000","last_undegraded":"2024-03-08T06:04:04.719812+0000","last_fullsized":"2024-03-08T06:04:04.719812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:52:36.740849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:04:10.804 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.055+0000 7f04b2400640 1 Processor -- start 2024-03-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.055+0000 7f04b2400640 1 -- start start 2024-03-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.056+0000 7f04b2400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04ac15df20 0x7f04ac0ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.056+0000 7f04b2400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04ac0d11b0 0x7f04ac162400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.056+0000 7f04b2400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ac15e300 con 0x7f04ac0b6bd0 2024-03-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.056+0000 7f04b2400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ac0d3990 con 0x7f04ac15df20 2024-03-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.056+0000 7f04b2400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac162940 con 0x7f04ac0d11b0 2024-03-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.057+0000 7f04b1000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04ac15df20 0x7f04ac0ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.057+0000 7f04b1000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f04ac15df20 0x7f04ac0ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45790/0 (socket says 172.21.15.40:45790) 2024-03-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.057+0000 7f04b1000640 1 -- 172.21.15.40:0/2805629868 learned_addr learned my addr 172.21.15.40:0/2805629868 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.057+0000 7f04b1a00640 1 --2- 172.21.15.40:0/2805629868 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04ac0d11b0 0x7f04ac162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.057+0000 7f04b1000640 1 -- 172.21.15.40:0/2805629868 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04ac0d11b0 msgr2=0x7f04ac162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.057+0000 7f04b1000640 1 --2- 172.21.15.40:0/2805629868 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04ac0d11b0 0x7f04ac162400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.057+0000 7f04b1000640 1 -- 172.21.15.40:0/2805629868 >> v1:172.21.15.40:6789/0 conn(0x7f04ac0b6bd0 legacy=0x7f04ac0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.057+0000 7f04b1000640 1 -- 172.21.15.40:0/2805629868 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0cf030 con 0x7f04ac15df20 2024-03-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.058+0000 7f04b1000640 1 --2- 172.21.15.40:0/2805629868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ac15df20 0x7f04ac0ce9e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f049c009870 tx=0x7f049c02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beab1f8f2ea05ec0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.059+0000 7f04ab400640 1 -- 172.21.15.40:0/2805629868 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c02f830 con 0x7f04ac15df20 2024-03-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.059+0000 7f04ab400640 1 -- 172.21.15.40:0/2805629868 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c02f9d0 con 0x7f04ac15df20 2024-03-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.059+0000 7f04ab400640 1 -- 172.21.15.40:0/2805629868 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c02fda0 con 0x7f04ac15df20 2024-03-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.059+0000 7f04b2400640 1 -- 172.21.15.40:0/2805629868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ac15df20 msgr2=0x7f04ac0ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.059+0000 7f04b2400640 1 --2- 172.21.15.40:0/2805629868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ac15df20 0x7f04ac0ce9e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f049c009870 tx=0x7f049c02ba10 comp rx=0 tx=0).stop 2024-03-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.060+0000 7f04b2400640 1 -- 172.21.15.40:0/2805629868 shutdown_connections 2024-03-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.060+0000 7f04b2400640 1 --2- 172.21.15.40:0/2805629868 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04ac0d11b0 0x7f04ac162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.060+0000 7f04b2400640 1 --2- 172.21.15.40:0/2805629868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ac15df20 0x7f04ac0ce9e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.060+0000 7f04b2400640 1 -- 172.21.15.40:0/2805629868 >> 172.21.15.40:0/2805629868 conn(0x7f04ac06c740 msgr2=0x7f04ac06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.060+0000 7f04b2400640 1 -- 172.21.15.40:0/2805629868 shutdown_connections 2024-03-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.060+0000 7f04b2400640 1 -- 172.21.15.40:0/2805629868 wait complete. 2024-03-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.062+0000 7f04b2400640 1 Processor -- start 2024-03-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.062+0000 7f04b2400640 1 -- start start 2024-03-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.063+0000 7f04b2400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04ac0b6bd0 0x7f04ac0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.063+0000 7f04b2400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ac0d11b0 0x7f04ac0b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.063+0000 7f04b2400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04ac15df20 0x7f04ac1455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.063+0000 7f04b2400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0d2180 con 0x7f04ac0d11b0 2024-03-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.063+0000 7f04b2400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0d1ea0 con 0x7f04ac15df20 2024-03-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.063+0000 7f04b2400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0d2460 con 0x7f04ac0b6bd0 2024-03-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.064+0000 7f04abe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ac0d11b0 0x7f04ac0b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.064+0000 7f04abe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ac0d11b0 0x7f04ac0b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45792/0 (socket says 172.21.15.40:45792) 2024-03-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.064+0000 7f04abe00640 1 -- 172.21.15.40:0/2749774194 learned_addr learned my addr 172.21.15.40:0/2749774194 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.064+0000 7f04abe00640 1 -- 172.21.15.40:0/2749774194 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04ac0b6bd0 msgr2=0x7f04ac0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.064+0000 7f04abe00640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04ac0b6bd0 0x7f04ac0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.064+0000 7f04abe00640 1 -- 172.21.15.40:0/2749774194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04ac15df20 msgr2=0x7f04ac1455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.064+0000 7f04abe00640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04ac15df20 0x7f04ac1455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.064+0000 7f04abe00640 1 -- 172.21.15.40:0/2749774194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0c78b0 con 0x7f04ac0d11b0 2024-03-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.064+0000 7f04abe00640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ac0d11b0 0x7f04ac0b56f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f049800c400 tx=0x7f049800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.065+0000 7f04a8c00640 1 -- 172.21.15.40:0/2749774194 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04980052d0 con 0x7f04ac0d11b0 2024-03-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.065+0000 7f04a8c00640 1 -- 172.21.15.40:0/2749774194 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0498005490 con 0x7f04ac0d11b0 2024-03-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.065+0000 7f04a8c00640 1 -- 172.21.15.40:0/2749774194 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498007260 con 0x7f04ac0d11b0 2024-03-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.065+0000 7f04b2400640 1 -- 172.21.15.40:0/2749774194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ac15fd50 con 0x7f04ac0d11b0 2024-03-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.065+0000 7f04b2400640 1 -- 172.21.15.40:0/2749774194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ac0c7b50 con 0x7f04ac0d11b0 2024-03-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.066+0000 7f04b2400640 1 -- 172.21.15.40:0/2749774194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ac075100 con 0x7f04ac0d11b0 2024-03-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.067+0000 7f04a8c00640 1 -- 172.21.15.40:0/2749774194 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0498007400 con 0x7f04ac0d11b0 2024-03-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.068+0000 7f04a8c00640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f048c06b520 0x7f048c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.068+0000 7f04a8c00640 1 -- 172.21.15.40:0/2749774194 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04980897a0 con 0x7f04ac0d11b0 2024-03-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.068+0000 7f04b1000640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f048c06b520 0x7f048c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.069+0000 7f04b1000640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f048c06b520 0x7f048c06d9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f049c036000 tx=0x7f049c039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.075+0000 7f04a8c00640 1 -- 172.21.15.40:0/2749774194 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0498058600 con 0x7f04ac0d11b0 2024-03-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.264+0000 7f04b2400640 1 -- 172.21.15.40:0/2749774194 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04ac0b2ea0 con 0x7f048c06b520 2024-03-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.283+0000 7f04a8c00640 1 -- 172.21.15.40:0/2749774194 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251874 (secure 0 0 0) 0x7f04ac0b2ea0 con 0x7f048c06b520 2024-03-08T06:04:11.287 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:11.289 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-03-08T06:04:11.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.288+0000 7f04b2400640 1 -- 172.21.15.40:0/2749774194 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f048c06b520 msgr2=0x7f048c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.288+0000 7f04b2400640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f048c06b520 0x7f048c06d9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f049c036000 tx=0x7f049c039040 comp rx=0 tx=0).stop 2024-03-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.288+0000 7f04b2400640 1 -- 172.21.15.40:0/2749774194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ac0d11b0 msgr2=0x7f04ac0b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.288+0000 7f04b2400640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ac0d11b0 0x7f04ac0b56f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f049800c400 tx=0x7f049800c8c0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.289+0000 7f04b2400640 1 -- 172.21.15.40:0/2749774194 shutdown_connections 2024-03-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.289+0000 7f04b2400640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f048c06b520 0x7f048c06d9e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.289+0000 7f04b2400640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04ac15df20 0x7f04ac1455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.289+0000 7f04b2400640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ac0d11b0 0x7f04ac0b56f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.289+0000 7f04b2400640 1 --2- 172.21.15.40:0/2749774194 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04ac0b6bd0 0x7f04ac0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.289+0000 7f04b2400640 1 -- 172.21.15.40:0/2749774194 >> 172.21.15.40:0/2749774194 conn(0x7f04ac06c740 msgr2=0x7f04ac0ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.289+0000 7f04b2400640 1 -- 172.21.15.40:0/2749774194 shutdown_connections 2024-03-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.289+0000 7f04b2400640 1 -- 172.21.15.40:0/2749774194 wait complete. 2024-03-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-08T06:04:09.484838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235956,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738764,"statfs":{"total":773094113280,"available":772852494336,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.968226"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618708+0000","last_change":"2024-03-08T06:04:02.552950+0000","last_active":"2024-03-08T06:04:03.618708+0000","last_peered":"2024-03-08T06:04:03.618708+0000","last_clean":"2024-03-08T06:04:03.618708+0000","last_became_active":"2024-03-08T06:04:02.552600+0000","last_became_peered":"2024-03-08T06:04:02.552600+0000","last_unstale":"2024-03-08T06:04:03.618708+0000","last_undegraded":"2024-03-08T06:04:03.618708+0000","last_fullsized":"2024-03-08T06:04:03.618708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:13:36.393811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553892+0000","last_change":"2024-03-08T06:04:02.547475+0000","last_active":"2024-03-08T06:04:03.553892+0000","last_peered":"2024-03-08T06:04:03.553892+0000","last_clean":"2024-03-08T06:04:03.553892+0000","last_became_active":"2024-03-08T06:04:02.545953+0000","last_became_peered":"2024-03-08T06:04:02.545953+0000","last_unstale":"2024-03-08T06:04:03.553892+0000","last_undegraded":"2024-03-08T06:04:03.553892+0000","last_fullsized":"2024-03-08T06:04:03.553892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719170+0000","last_change":"2024-03-08T06:04:02.544781+0000","last_active":"2024-03-08T06:04:04.719170+0000","last_peered":"2024-03-08T06:04:04.719170+0000","last_clean":"2024-03-08T06:04:04.719170+0000","last_became_active":"2024-03-08T06:04:02.543995+0000","last_became_peered":"2024-03-08T06:04:02.543995+0000","last_unstale":"2024-03-08T06:04:04.719170+0000","last_undegraded":"2024-03-08T06:04:04.719170+0000","last_fullsized":"2024-03-08T06:04:04.719170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515851+0000","last_change":"2024-03-08T06:04:02.551644+0000","last_active":"2024-03-08T06:04:03.515851+0000","last_peered":"2024-03-08T06:04:03.515851+0000","last_clean":"2024-03-08T06:04:03.515851+0000","last_became_active":"2024-03-08T06:04:02.550930+0000","last_became_peered":"2024-03-08T06:04:02.550930+0000","last_unstale":"2024-03-08T06:04:03.515851+0000","last_undegraded":"2024-03-08T06:04:03.515851+0000","last_fullsized":"2024-03-08T06:04:03.515851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:23:55.655734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617514+0000","last_change":"2024-03-08T06:04:02.545560+0000","last_active":"2024-03-08T06:04:03.617514+0000","last_peered":"2024-03-08T06:04:03.617514+0000","last_clean":"2024-03-08T06:04:03.617514+0000","last_became_active":"2024-03-08T06:04:02.543855+0000","last_became_peered":"2024-03-08T06:04:02.543855+0000","last_unstale":"2024-03-08T06:04:03.617514+0000","last_undegraded":"2024-03-08T06:04:03.617514+0000","last_fullsized":"2024-03-08T06:04:03.617514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515916+0000","last_change":"2024-03-08T06:04:02.550172+0000","last_active":"2024-03-08T06:04:03.515916+0000","last_peered":"2024-03-08T06:04:03.515916+0000","last_clean":"2024-03-08T06:04:03.515916+0000","last_became_active":"2024-03-08T06:04:02.549807+0000","last_became_peered":"2024-03-08T06:04:02.549807+0000","last_unstale":"2024-03-08T06:04:03.515916+0000","last_undegraded":"2024-03-08T06:04:03.515916+0000","last_fullsized":"2024-03-08T06:04:03.515916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617257+0000","last_change":"2024-03-08T06:04:02.547691+0000","last_active":"2024-03-08T06:04:03.617257+0000","last_peered":"2024-03-08T06:04:03.617257+0000","last_clean":"2024-03-08T06:04:03.617257+0000","last_became_active":"2024-03-08T06:04:02.547325+0000","last_became_peered":"2024-03-08T06:04:02.547325+0000","last_unstale":"2024-03-08T06:04:03.617257+0000","last_undegraded":"2024-03-08T06:04:03.617257+0000","last_fullsized":"2024-03-08T06:04:03.617257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511377+0000","last_change":"2024-03-08T06:04:02.553110+0000","last_active":"2024-03-08T06:04:03.511377+0000","last_peered":"2024-03-08T06:04:03.511377+0000","last_clean":"2024-03-08T06:04:03.511377+0000","last_became_active":"2024-03-08T06:04:02.552635+0000","last_became_peered":"2024-03-08T06:04:02.552635+0000","last_unstale":"2024-03-08T06:04:03.511377+0000","last_undegraded":"2024-03-08T06:04:03.511377+0000","last_fullsized":"2024-03-08T06:04:03.511377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618751+0000","last_change":"2024-03-08T06:04:02.570801+0000","last_active":"2024-03-08T06:04:03.618751+0000","last_peered":"2024-03-08T06:04:03.618751+0000","last_clean":"2024-03-08T06:04:03.618751+0000","last_became_active":"2024-03-08T06:04:02.570558+0000","last_became_peered":"2024-03-08T06:04:02.570558+0000","last_unstale":"2024-03-08T06:04:03.618751+0000","last_undegraded":"2024-03-08T06:04:03.618751+0000","last_fullsized":"2024-03-08T06:04:03.618751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:07:57.853863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559891+0000","last_change":"2024-03-08T06:04:02.562132+0000","last_active":"2024-03-08T06:04:03.559891+0000","last_peered":"2024-03-08T06:04:03.559891+0000","last_clean":"2024-03-08T06:04:03.559891+0000","last_became_active":"2024-03-08T06:04:02.561851+0000","last_became_peered":"2024-03-08T06:04:02.561851+0000","last_unstale":"2024-03-08T06:04:03.559891+0000","last_undegraded":"2024-03-08T06:04:03.559891+0000","last_fullsized":"2024-03-08T06:04:03.559891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559579+0000","last_change":"2024-03-08T06:04:02.546846+0000","last_active":"2024-03-08T06:04:03.559579+0000","last_peered":"2024-03-08T06:04:03.559579+0000","last_clean":"2024-03-08T06:04:03.559579+0000","last_became_active":"2024-03-08T06:04:02.546522+0000","last_became_peered":"2024-03-08T06:04:02.546522+0000","last_unstale":"2024-03-08T06:04:03.559579+0000","last_undegraded":"2024-03-08T06:04:03.559579+0000","last_fullsized":"2024-03-08T06:04:03.559579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118422+0000","last_change":"2024-03-08T06:04:02.555942+0000","last_active":"2024-03-08T06:04:04.118422+0000","last_peered":"2024-03-08T06:04:04.118422+0000","last_clean":"2024-03-08T06:04:04.118422+0000","last_became_active":"2024-03-08T06:04:02.555651+0000","last_became_peered":"2024-03-08T06:04:02.555651+0000","last_unstale":"2024-03-08T06:04:04.118422+0000","last_undegraded":"2024-03-08T06:04:04.118422+0000","last_fullsized":"2024-03-08T06:04:04.118422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:57.032278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511828+0000","last_change":"2024-03-08T06:04:02.551969+0000","last_active":"2024-03-08T06:04:03.511828+0000","last_peered":"2024-03-08T06:04:03.511828+0000","last_clean":"2024-03-08T06:04:03.511828+0000","last_became_active":"2024-03-08T06:04:02.551664+0000","last_became_peered":"2024-03-08T06:04:02.551664+0000","last_unstale":"2024-03-08T06:04:03.511828+0000","last_undegraded":"2024-03-08T06:04:03.511828+0000","last_fullsized":"2024-03-08T06:04:03.511828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.718486+0000","last_change":"2024-03-08T06:04:02.542608+0000","last_active":"2024-03-08T06:04:04.718486+0000","last_peered":"2024-03-08T06:04:04.718486+0000","last_clean":"2024-03-08T06:04:04.718486+0000","last_became_active":"2024-03-08T06:04:02.542318+0000","last_became_peered":"2024-03-08T06:04:02.542318+0000","last_unstale":"2024-03-08T06:04:04.718486+0000","last_undegraded":"2024-03-08T06:04:04.718486+0000","last_fullsized":"2024-03-08T06:04:04.718486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:04:14.473084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553245+0000","last_change":"2024-03-08T06:04:02.548664+0000","last_active":"2024-03-08T06:04:03.553245+0000","last_peered":"2024-03-08T06:04:03.553245+0000","last_clean":"2024-03-08T06:04:03.553245+0000","last_became_active":"2024-03-08T06:04:02.547601+0000","last_became_peered":"2024-03-08T06:04:02.547601+0000","last_unstale":"2024-03-08T06:04:03.553245+0000","last_undegraded":"2024-03-08T06:04:03.553245+0000","last_fullsized":"2024-03-08T06:04:03.553245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515232+0000","last_change":"2024-03-08T06:04:02.551648+0000","last_active":"2024-03-08T06:04:03.515232+0000","last_peered":"2024-03-08T06:04:03.515232+0000","last_clean":"2024-03-08T06:04:03.515232+0000","last_became_active":"2024-03-08T06:04:02.551290+0000","last_became_peered":"2024-03-08T06:04:02.551290+0000","last_unstale":"2024-03-08T06:04:03.515232+0000","last_undegraded":"2024-03-08T06:04:03.515232+0000","last_fullsized":"2024-03-08T06:04:03.515232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.518022+0000","last_change":"2024-03-08T06:04:02.556704+0000","last_active":"2024-03-08T06:04:03.518022+0000","last_peered":"2024-03-08T06:04:03.518022+0000","last_clean":"2024-03-08T06:04:03.518022+0000","last_became_active":"2024-03-08T06:04:02.556429+0000","last_became_peered":"2024-03-08T06:04:02.556429+0000","last_unstale":"2024-03-08T06:04:03.518022+0000","last_undegraded":"2024-03-08T06:04:03.518022+0000","last_fullsized":"2024-03-08T06:04:03.518022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:13:36.393811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.697123+0000","last_change":"2024-03-08T06:04:02.539066+0000","last_active":"2024-03-08T06:04:04.697123+0000","last_peered":"2024-03-08T06:04:04.697123+0000","last_clean":"2024-03-08T06:04:04.697123+0000","last_became_active":"2024-03-08T06:04:02.538745+0000","last_became_peered":"2024-03-08T06:04:02.538745+0000","last_unstale":"2024-03-08T06:04:04.697123+0000","last_undegraded":"2024-03-08T06:04:04.697123+0000","last_fullsized":"2024-03-08T06:04:04.697123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:06:00.048492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618590+0000","last_change":"2024-03-08T06:04:02.551987+0000","last_active":"2024-03-08T06:04:03.618590+0000","last_peered":"2024-03-08T06:04:03.618590+0000","last_clean":"2024-03-08T06:04:03.618590+0000","last_became_active":"2024-03-08T06:04:02.551425+0000","last_became_peered":"2024-03-08T06:04:02.551425+0000","last_unstale":"2024-03-08T06:04:03.618590+0000","last_undegraded":"2024-03-08T06:04:03.618590+0000","last_fullsized":"2024-03-08T06:04:03.618590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:41:43.397781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559135+0000","last_change":"2024-03-08T06:04:02.559334+0000","last_active":"2024-03-08T06:04:03.559135+0000","last_peered":"2024-03-08T06:04:03.559135+0000","last_clean":"2024-03-08T06:04:03.559135+0000","last_became_active":"2024-03-08T06:04:02.558979+0000","last_became_peered":"2024-03-08T06:04:02.558979+0000","last_unstale":"2024-03-08T06:04:03.559135+0000","last_undegraded":"2024-03-08T06:04:03.559135+0000","last_fullsized":"2024-03-08T06:04:03.559135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719096+0000","last_change":"2024-03-08T06:04:02.545858+0000","last_active":"2024-03-08T06:04:04.719096+0000","last_peered":"2024-03-08T06:04:04.719096+0000","last_clean":"2024-03-08T06:04:04.719096+0000","last_became_active":"2024-03-08T06:04:02.545518+0000","last_became_peered":"2024-03-08T06:04:02.545518+0000","last_unstale":"2024-03-08T06:04:04.719096+0000","last_undegraded":"2024-03-08T06:04:04.719096+0000","last_fullsized":"2024-03-08T06:04:04.719096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516005+0000","last_change":"2024-03-08T06:04:02.553462+0000","last_active":"2024-03-08T06:04:03.516005+0000","last_peered":"2024-03-08T06:04:03.516005+0000","last_clean":"2024-03-08T06:04:03.516005+0000","last_became_active":"2024-03-08T06:04:02.552688+0000","last_became_peered":"2024-03-08T06:04:02.552688+0000","last_unstale":"2024-03-08T06:04:03.516005+0000","last_undegraded":"2024-03-08T06:04:03.516005+0000","last_fullsized":"2024-03-08T06:04:03.516005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511690+0000","last_change":"2024-03-08T06:04:02.560361+0000","last_active":"2024-03-08T06:04:03.511690+0000","last_peered":"2024-03-08T06:04:03.511690+0000","last_clean":"2024-03-08T06:04:03.511690+0000","last_became_active":"2024-03-08T06:04:02.560083+0000","last_became_peered":"2024-03-08T06:04:02.560083+0000","last_unstale":"2024-03-08T06:04:03.511690+0000","last_undegraded":"2024-03-08T06:04:03.511690+0000","last_fullsized":"2024-03-08T06:04:03.511690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515403+0000","last_change":"2024-03-08T06:04:02.549163+0000","last_active":"2024-03-08T06:04:03.515403+0000","last_peered":"2024-03-08T06:04:03.515403+0000","last_clean":"2024-03-08T06:04:03.515403+0000","last_became_active":"2024-03-08T06:04:02.548715+0000","last_became_peered":"2024-03-08T06:04:02.548715+0000","last_unstale":"2024-03-08T06:04:03.515403+0000","last_undegraded":"2024-03-08T06:04:03.515403+0000","last_fullsized":"2024-03-08T06:04:03.515403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512314+0000","last_change":"2024-03-08T06:04:02.565662+0000","last_active":"2024-03-08T06:04:03.512314+0000","last_peered":"2024-03-08T06:04:03.512314+0000","last_clean":"2024-03-08T06:04:03.512314+0000","last_became_active":"2024-03-08T06:04:02.565344+0000","last_became_peered":"2024-03-08T06:04:02.565344+0000","last_unstale":"2024-03-08T06:04:03.512314+0000","last_undegraded":"2024-03-08T06:04:03.512314+0000","last_fullsized":"2024-03-08T06:04:03.512314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:23:55.655734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.560225+0000","last_change":"2024-03-08T06:04:02.545865+0000","last_active":"2024-03-08T06:04:03.560225+0000","last_peered":"2024-03-08T06:04:03.560225+0000","last_clean":"2024-03-08T06:04:03.560225+0000","last_became_active":"2024-03-08T06:04:02.543828+0000","last_became_peered":"2024-03-08T06:04:02.543828+0000","last_unstale":"2024-03-08T06:04:03.560225+0000","last_undegraded":"2024-03-08T06:04:03.560225+0000","last_fullsized":"2024-03-08T06:04:03.560225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553769+0000","last_change":"2024-03-08T06:04:02.544859+0000","last_active":"2024-03-08T06:04:03.553769+0000","last_peered":"2024-03-08T06:04:03.553769+0000","last_clean":"2024-03-08T06:04:03.553769+0000","last_became_active":"2024-03-08T06:04:02.544502+0000","last_became_peered":"2024-03-08T06:04:02.544502+0000","last_unstale":"2024-03-08T06:04:03.553769+0000","last_undegraded":"2024-03-08T06:04:03.553769+0000","last_fullsized":"2024-03-08T06:04:03.553769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512315+0000","last_change":"2024-03-08T06:04:02.563733+0000","last_active":"2024-03-08T06:04:03.512315+0000","last_peered":"2024-03-08T06:04:03.512315+0000","last_clean":"2024-03-08T06:04:03.512315+0000","last_became_active":"2024-03-08T06:04:02.563420+0000","last_became_peered":"2024-03-08T06:04:02.563420+0000","last_unstale":"2024-03-08T06:04:03.512315+0000","last_undegraded":"2024-03-08T06:04:03.512315+0000","last_fullsized":"2024-03-08T06:04:03.512315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:35:18.730168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719077+0000","last_change":"2024-03-08T06:04:02.548150+0000","last_active":"2024-03-08T06:04:04.719077+0000","last_peered":"2024-03-08T06:04:04.719077+0000","last_clean":"2024-03-08T06:04:04.719077+0000","last_became_active":"2024-03-08T06:04:02.547846+0000","last_became_peered":"2024-03-08T06:04:02.547846+0000","last_unstale":"2024-03-08T06:04:04.719077+0000","last_undegraded":"2024-03-08T06:04:04.719077+0000","last_fullsized":"2024-03-08T06:04:04.719077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516515+0000","last_change":"2024-03-08T06:04:02.547204+0000","last_active":"2024-03-08T06:04:03.516515+0000","last_peered":"2024-03-08T06:04:03.516515+0000","last_clean":"2024-03-08T06:04:03.516515+0000","last_became_active":"2024-03-08T06:04:02.546850+0000","last_became_peered":"2024-03-08T06:04:02.546850+0000","last_unstale":"2024-03-08T06:04:03.516515+0000","last_undegraded":"2024-03-08T06:04:03.516515+0000","last_fullsized":"2024-03-08T06:04:03.516515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:32:53.947147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.696799+0000","last_change":"2024-03-08T06:04:02.553107+0000","last_active":"2024-03-08T06:04:04.696799+0000","last_peered":"2024-03-08T06:04:04.696799+0000","last_clean":"2024-03-08T06:04:04.696799+0000","last_became_active":"2024-03-08T06:04:02.552862+0000","last_became_peered":"2024-03-08T06:04:02.552862+0000","last_unstale":"2024-03-08T06:04:04.696799+0000","last_undegraded":"2024-03-08T06:04:04.696799+0000","last_fullsized":"2024-03-08T06:04:04.696799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:57.032278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559013+0000","last_change":"2024-03-08T06:04:02.544639+0000","last_active":"2024-03-08T06:04:03.559013+0000","last_peered":"2024-03-08T06:04:03.559013+0000","last_clean":"2024-03-08T06:04:03.559013+0000","last_became_active":"2024-03-08T06:04:02.544193+0000","last_became_peered":"2024-03-08T06:04:02.544193+0000","last_unstale":"2024-03-08T06:04:03.559013+0000","last_undegraded":"2024-03-08T06:04:03.559013+0000","last_fullsized":"2024-03-08T06:04:03.559013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.117892+0000","last_change":"2024-03-08T06:04:02.558328+0000","last_active":"2024-03-08T06:04:04.117892+0000","last_peered":"2024-03-08T06:04:04.117892+0000","last_clean":"2024-03-08T06:04:04.117892+0000","last_became_active":"2024-03-08T06:04:02.555015+0000","last_became_peered":"2024-03-08T06:04:02.555015+0000","last_unstale":"2024-03-08T06:04:04.117892+0000","last_undegraded":"2024-03-08T06:04:04.117892+0000","last_fullsized":"2024-03-08T06:04:04.117892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:52.535460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512099+0000","last_change":"2024-03-08T06:04:01.555471+0000","last_active":"2024-03-08T06:04:03.512099+0000","last_peered":"2024-03-08T06:04:03.512099+0000","last_clean":"2024-03-08T06:04:03.512099+0000","last_became_active":"2024-03-08T06:04:01.555171+0000","last_became_peered":"2024-03-08T06:04:01.555171+0000","last_unstale":"2024-03-08T06:04:03.512099+0000","last_undegraded":"2024-03-08T06:04:03.512099+0000","last_fullsized":"2024-03-08T06:04:03.512099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719499+0000","last_change":"2024-03-08T06:04:02.563404+0000","last_active":"2024-03-08T06:04:04.719499+0000","last_peered":"2024-03-08T06:04:04.719499+0000","last_clean":"2024-03-08T06:04:04.719499+0000","last_became_active":"2024-03-08T06:04:02.558959+0000","last_became_peered":"2024-03-08T06:04:02.558959+0000","last_unstale":"2024-03-08T06:04:04.719499+0000","last_undegraded":"2024-03-08T06:04:04.719499+0000","last_fullsized":"2024-03-08T06:04:04.719499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559117+0000","last_change":"2024-03-08T06:04:01.549647+0000","last_active":"2024-03-08T06:04:03.559117+0000","last_peered":"2024-03-08T06:04:03.559117+0000","last_clean":"2024-03-08T06:04:03.559117+0000","last_became_active":"2024-03-08T06:04:01.549345+0000","last_became_peered":"2024-03-08T06:04:01.549345+0000","last_unstale":"2024-03-08T06:04:03.559117+0000","last_undegraded":"2024-03-08T06:04:03.559117+0000","last_fullsized":"2024-03-08T06:04:03.559117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511841+0000","last_change":"2024-03-08T06:04:01.551862+0000","last_active":"2024-03-08T06:04:03.511841+0000","last_peered":"2024-03-08T06:04:03.511841+0000","last_clean":"2024-03-08T06:04:03.511841+0000","last_became_active":"2024-03-08T06:04:01.551606+0000","last_became_peered":"2024-03-08T06:04:01.551606+0000","last_unstale":"2024-03-08T06:04:03.511841+0000","last_undegraded":"2024-03-08T06:04:03.511841+0000","last_fullsized":"2024-03-08T06:04:03.511841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:52:36.740849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512342+0000","last_change":"2024-03-08T06:04:02.548972+0000","last_active":"2024-03-08T06:04:03.512342+0000","last_peered":"2024-03-08T06:04:03.512342+0000","last_clean":"2024-03-08T06:04:03.512342+0000","last_became_active":"2024-03-08T06:04:02.548136+0000","last_became_peered":"2024-03-08T06:04:02.548136+0000","last_unstale":"2024-03-08T06:04:03.512342+0000","last_undegraded":"2024-03-08T06:04:03.512342+0000","last_fullsized":"2024-03-08T06:04:03.512342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516175+0000","last_change":"2024-03-08T06:04:01.556755+0000","last_active":"2024-03-08T06:04:03.516175+0000","last_peered":"2024-03-08T06:04:03.516175+0000","last_clean":"2024-03-08T06:04:03.516175+0000","last_became_active":"2024-03-08T06:04:01.555842+0000","last_became_peered":"2024-03-08T06:04:01.555842+0000","last_unstale":"2024-03-08T06:04:03.516175+0000","last_undegraded":"2024-03-08T06:04:03.516175+0000","last_fullsized":"2024-03-08T06:04:03.516175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553244+0000","last_change":"2024-03-08T06:04:02.549137+0000","last_active":"2024-03-08T06:04:03.553244+0000","last_peered":"2024-03-08T06:04:03.553244+0000","last_clean":"2024-03-08T06:04:03.553244+0000","last_became_active":"2024-03-08T06:04:02.547990+0000","last_became_peered":"2024-03-08T06:04:02.547990+0000","last_unstale":"2024-03-08T06:04:03.553244+0000","last_undegraded":"2024-03-08T06:04:03.553244+0000","last_fullsized":"2024-03-08T06:04:03.553244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559731+0000","last_change":"2024-03-08T06:04:01.555948+0000","last_active":"2024-03-08T06:04:03.559731+0000","last_peered":"2024-03-08T06:04:03.559731+0000","last_clean":"2024-03-08T06:04:03.559731+0000","last_became_active":"2024-03-08T06:04:01.555666+0000","last_became_peered":"2024-03-08T06:04:01.555666+0000","last_unstale":"2024-03-08T06:04:03.559731+0000","last_undegraded":"2024-03-08T06:04:03.559731+0000","last_fullsized":"2024-03-08T06:04:03.559731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517528+0000","last_change":"2024-03-08T06:04:02.558089+0000","last_active":"2024-03-08T06:04:03.517528+0000","last_peered":"2024-03-08T06:04:03.517528+0000","last_clean":"2024-03-08T06:04:03.517528+0000","last_became_active":"2024-03-08T06:04:02.557697+0000","last_became_peered":"2024-03-08T06:04:02.557697+0000","last_unstale":"2024-03-08T06:04:03.517528+0000","last_undegraded":"2024-03-08T06:04:03.517528+0000","last_fullsized":"2024-03-08T06:04:03.517528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:19:21.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517312+0000","last_change":"2024-03-08T06:04:01.559428+0000","last_active":"2024-03-08T06:04:03.517312+0000","last_peered":"2024-03-08T06:04:03.517312+0000","last_clean":"2024-03-08T06:04:03.517312+0000","last_became_active":"2024-03-08T06:04:01.559156+0000","last_became_peered":"2024-03-08T06:04:01.559156+0000","last_unstale":"2024-03-08T06:04:03.517312+0000","last_undegraded":"2024-03-08T06:04:03.517312+0000","last_fullsized":"2024-03-08T06:04:03.517312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719809+0000","last_change":"2024-03-08T06:04:02.551412+0000","last_active":"2024-03-08T06:04:04.719809+0000","last_peered":"2024-03-08T06:04:04.719809+0000","last_clean":"2024-03-08T06:04:04.719809+0000","last_became_active":"2024-03-08T06:04:02.550080+0000","last_became_peered":"2024-03-08T06:04:02.550080+0000","last_unstale":"2024-03-08T06:04:04.719809+0000","last_undegraded":"2024-03-08T06:04:04.719809+0000","last_fullsized":"2024-03-08T06:04:04.719809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553697+0000","last_change":"2024-03-08T06:04:01.554654+0000","last_active":"2024-03-08T06:04:03.553697+0000","last_peered":"2024-03-08T06:04:03.553697+0000","last_clean":"2024-03-08T06:04:03.553697+0000","last_became_active":"2024-03-08T06:04:01.554238+0000","last_became_peered":"2024-03-08T06:04:01.554238+0000","last_unstale":"2024-03-08T06:04:03.553697+0000","last_undegraded":"2024-03-08T06:04:03.553697+0000","last_fullsized":"2024-03-08T06:04:03.553697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118544+0000","last_change":"2024-03-08T06:04:02.556415+0000","last_active":"2024-03-08T06:04:04.118544+0000","last_peered":"2024-03-08T06:04:04.118544+0000","last_clean":"2024-03-08T06:04:04.118544+0000","last_became_active":"2024-03-08T06:04:02.556055+0000","last_became_peered":"2024-03-08T06:04:02.556055+0000","last_unstale":"2024-03-08T06:04:04.118544+0000","last_undegraded":"2024-03-08T06:04:04.118544+0000","last_fullsized":"2024-03-08T06:04:04.118544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:06:00.048492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559371+0000","last_change":"2024-03-08T06:04:01.549872+0000","last_active":"2024-03-08T06:04:03.559371+0000","last_peered":"2024-03-08T06:04:03.559371+0000","last_clean":"2024-03-08T06:04:03.559371+0000","last_became_active":"2024-03-08T06:04:01.549313+0000","last_became_peered":"2024-03-08T06:04:01.549313+0000","last_unstale":"2024-03-08T06:04:03.559371+0000","last_undegraded":"2024-03-08T06:04:03.559371+0000","last_fullsized":"2024-03-08T06:04:03.559371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618023+0000","last_change":"2024-03-08T06:04:02.549539+0000","last_active":"2024-03-08T06:04:03.618023+0000","last_peered":"2024-03-08T06:04:03.618023+0000","last_clean":"2024-03-08T06:04:03.618023+0000","last_became_active":"2024-03-08T06:04:02.549245+0000","last_became_peered":"2024-03-08T06:04:02.549245+0000","last_unstale":"2024-03-08T06:04:03.618023+0000","last_undegraded":"2024-03-08T06:04:03.618023+0000","last_fullsized":"2024-03-08T06:04:03.618023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:33:11.525956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719392+0000","last_change":"2024-03-08T06:04:01.542685+0000","last_active":"2024-03-08T06:04:04.719392+0000","last_peered":"2024-03-08T06:04:04.719392+0000","last_clean":"2024-03-08T06:04:04.719392+0000","last_became_active":"2024-03-08T06:04:01.542236+0000","last_became_peered":"2024-03-08T06:04:01.542236+0000","last_unstale":"2024-03-08T06:04:04.719392+0000","last_undegraded":"2024-03-08T06:04:04.719392+0000","last_fullsized":"2024-03-08T06:04:04.719392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:35:24.125408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.718718+0000","last_change":"2024-03-08T06:04:02.553210+0000","last_active":"2024-03-08T06:04:04.718718+0000","last_peered":"2024-03-08T06:04:04.718718+0000","last_clean":"2024-03-08T06:04:04.718718+0000","last_became_active":"2024-03-08T06:04:02.552889+0000","last_became_peered":"2024-03-08T06:04:02.552889+0000","last_unstale":"2024-03-08T06:04:04.718718+0000","last_undegraded":"2024-03-08T06:04:04.718718+0000","last_fullsized":"2024-03-08T06:04:04.718718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:28:17.039893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516402+0000","last_change":"2024-03-08T06:04:01.554506+0000","last_active":"2024-03-08T06:04:03.516402+0000","last_peered":"2024-03-08T06:04:03.516402+0000","last_clean":"2024-03-08T06:04:03.516402+0000","last_became_active":"2024-03-08T06:04:01.554224+0000","last_became_peered":"2024-03-08T06:04:01.554224+0000","last_unstale":"2024-03-08T06:04:03.516402+0000","last_undegraded":"2024-03-08T06:04:03.516402+0000","last_fullsized":"2024-03-08T06:04:03.516402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617833+0000","last_change":"2024-03-08T06:04:02.560754+0000","last_active":"2024-03-08T06:04:03.617833+0000","last_peered":"2024-03-08T06:04:03.617833+0000","last_clean":"2024-03-08T06:04:03.617833+0000","last_became_active":"2024-03-08T06:04:02.560482+0000","last_became_peered":"2024-03-08T06:04:02.560482+0000","last_unstale":"2024-03-08T06:04:03.617833+0000","last_undegraded":"2024-03-08T06:04:03.617833+0000","last_fullsized":"2024-03-08T06:04:03.617833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:32:53.947147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559130+0000","last_change":"2024-03-08T06:04:01.545961+0000","last_active":"2024-03-08T06:04:03.559130+0000","last_peered":"2024-03-08T06:04:03.559130+0000","last_clean":"2024-03-08T06:04:03.559130+0000","last_became_active":"2024-03-08T06:04:01.545579+0000","last_became_peered":"2024-03-08T06:04:01.545579+0000","last_unstale":"2024-03-08T06:04:03.559130+0000","last_undegraded":"2024-03-08T06:04:03.559130+0000","last_fullsized":"2024-03-08T06:04:03.559130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.558973+0000","last_change":"2024-03-08T06:04:02.544735+0000","last_active":"2024-03-08T06:04:03.558973+0000","last_peered":"2024-03-08T06:04:03.558973+0000","last_clean":"2024-03-08T06:04:03.558973+0000","last_became_active":"2024-03-08T06:04:02.544438+0000","last_became_peered":"2024-03-08T06:04:02.544438+0000","last_unstale":"2024-03-08T06:04:03.558973+0000","last_undegraded":"2024-03-08T06:04:03.558973+0000","last_fullsized":"2024-03-08T06:04:03.558973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:11:04.612788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.696817+0000","last_change":"2024-03-08T06:04:01.541060+0000","last_active":"2024-03-08T06:04:04.696817+0000","last_peered":"2024-03-08T06:04:04.696817+0000","last_clean":"2024-03-08T06:04:04.696817+0000","last_became_active":"2024-03-08T06:04:01.540657+0000","last_became_peered":"2024-03-08T06:04:01.540657+0000","last_unstale":"2024-03-08T06:04:04.696817+0000","last_undegraded":"2024-03-08T06:04:04.696817+0000","last_fullsized":"2024-03-08T06:04:04.696817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:38:32.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617381+0000","last_change":"2024-03-08T06:04:02.565226+0000","last_active":"2024-03-08T06:04:03.617381+0000","last_peered":"2024-03-08T06:04:03.617381+0000","last_clean":"2024-03-08T06:04:03.617381+0000","last_became_active":"2024-03-08T06:04:02.564692+0000","last_became_peered":"2024-03-08T06:04:02.564692+0000","last_unstale":"2024-03-08T06:04:03.617381+0000","last_undegraded":"2024-03-08T06:04:03.617381+0000","last_fullsized":"2024-03-08T06:04:03.617381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118978+0000","last_change":"2024-03-08T06:04:01.540655+0000","last_active":"2024-03-08T06:04:04.118978+0000","last_peered":"2024-03-08T06:04:04.118978+0000","last_clean":"2024-03-08T06:04:04.118978+0000","last_became_active":"2024-03-08T06:04:01.540205+0000","last_became_peered":"2024-03-08T06:04:01.540205+0000","last_unstale":"2024-03-08T06:04:04.118978+0000","last_undegraded":"2024-03-08T06:04:04.118978+0000","last_fullsized":"2024-03-08T06:04:04.118978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:12:32.122620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618419+0000","last_change":"2024-03-08T06:04:02.559001+0000","last_active":"2024-03-08T06:04:03.618419+0000","last_peered":"2024-03-08T06:04:03.618419+0000","last_clean":"2024-03-08T06:04:03.618419+0000","last_became_active":"2024-03-08T06:04:02.558681+0000","last_became_peered":"2024-03-08T06:04:02.558681+0000","last_unstale":"2024-03-08T06:04:03.618419+0000","last_undegraded":"2024-03-08T06:04:03.618419+0000","last_fullsized":"2024-03-08T06:04:03.618419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:19:21.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.696931+0000","last_change":"2024-03-08T06:04:01.542669+0000","last_active":"2024-03-08T06:04:04.696931+0000","last_peered":"2024-03-08T06:04:04.696931+0000","last_clean":"2024-03-08T06:04:04.696931+0000","last_became_active":"2024-03-08T06:04:01.540916+0000","last_became_peered":"2024-03-08T06:04:01.540916+0000","last_unstale":"2024-03-08T06:04:04.696931+0000","last_undegraded":"2024-03-08T06:04:04.696931+0000","last_fullsized":"2024-03-08T06:04:04.696931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618960+0000","last_change":"2024-03-08T06:04:02.569733+0000","last_active":"2024-03-08T06:04:03.618960+0000","last_peered":"2024-03-08T06:04:03.618960+0000","last_clean":"2024-03-08T06:04:03.618960+0000","last_became_active":"2024-03-08T06:04:02.567299+0000","last_became_peered":"2024-03-08T06:04:02.567299+0000","last_unstale":"2024-03-08T06:04:03.618960+0000","last_undegraded":"2024-03-08T06:04:03.618960+0000","last_fullsized":"2024-03-08T06:04:03.618960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:16:15.668237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553839+0000","last_change":"2024-03-08T06:04:01.544281+0000","last_active":"2024-03-08T06:04:03.553839+0000","last_peered":"2024-03-08T06:04:03.553839+0000","last_clean":"2024-03-08T06:04:03.553839+0000","last_became_active":"2024-03-08T06:04:01.543982+0000","last_became_peered":"2024-03-08T06:04:01.543982+0000","last_unstale":"2024-03-08T06:04:03.553839+0000","last_undegraded":"2024-03-08T06:04:03.553839+0000","last_fullsized":"2024-03-08T06:04:03.553839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:50.471516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516800+0000","last_change":"2024-03-08T06:04:02.551888+0000","last_active":"2024-03-08T06:04:03.516800+0000","last_peered":"2024-03-08T06:04:03.516800+0000","last_clean":"2024-03-08T06:04:03.516800+0000","last_became_active":"2024-03-08T06:04:02.551321+0000","last_became_peered":"2024-03-08T06:04:02.551321+0000","last_unstale":"2024-03-08T06:04:03.516800+0000","last_undegraded":"2024-03-08T06:04:03.516800+0000","last_fullsized":"2024-03-08T06:04:03.516800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:33:11.525956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.696683+0000","last_change":"2024-03-08T06:04:01.554656+0000","last_active":"2024-03-08T06:04:04.696683+0000","last_peered":"2024-03-08T06:04:04.696683+0000","last_clean":"2024-03-08T06:04:04.696683+0000","last_became_active":"2024-03-08T06:04:01.554326+0000","last_became_peered":"2024-03-08T06:04:01.554326+0000","last_unstale":"2024-03-08T06:04:04.696683+0000","last_undegraded":"2024-03-08T06:04:04.696683+0000","last_fullsized":"2024-03-08T06:04:04.696683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618096+0000","last_change":"2024-03-08T06:04:02.568164+0000","last_active":"2024-03-08T06:04:03.618096+0000","last_peered":"2024-03-08T06:04:03.618096+0000","last_clean":"2024-03-08T06:04:03.618096+0000","last_became_active":"2024-03-08T06:04:02.567123+0000","last_became_peered":"2024-03-08T06:04:02.567123+0000","last_unstale":"2024-03-08T06:04:03.618096+0000","last_undegraded":"2024-03-08T06:04:03.618096+0000","last_fullsized":"2024-03-08T06:04:03.618096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:42:12.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617235+0000","last_change":"2024-03-08T06:03:57.473597+0000","last_active":"2024-03-08T06:04:03.617235+0000","last_peered":"2024-03-08T06:04:03.617235+0000","last_clean":"2024-03-08T06:04:03.617235+0000","last_became_active":"2024-03-08T06:03:57.468593+0000","last_became_peered":"2024-03-08T06:03:57.468593+0000","last_unstale":"2024-03-08T06:04:03.617235+0000","last_undegraded":"2024-03-08T06:04:03.617235+0000","last_fullsized":"2024-03-08T06:04:03.617235+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_clean_scrub_stamp":"2024-03-08T06:03:56.214441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:45:28.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.515695+0000","last_change":"2024-03-08T06:04:01.554640+0000","last_active":"2024-03-08T06:04:03.515695+0000","last_peered":"2024-03-08T06:04:03.515695+0000","last_clean":"2024-03-08T06:04:03.515695+0000","last_became_active":"2024-03-08T06:04:01.554274+0000","last_became_peered":"2024-03-08T06:04:01.554274+0000","last_unstale":"2024-03-08T06:04:03.515695+0000","last_undegraded":"2024-03-08T06:04:03.515695+0000","last_fullsized":"2024-03-08T06:04:03.515695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511891+0000","last_change":"2024-03-08T06:04:02.556830+0000","last_active":"2024-03-08T06:04:03.511891+0000","last_peered":"2024-03-08T06:04:03.511891+0000","last_clean":"2024-03-08T06:04:03.511891+0000","last_became_active":"2024-03-08T06:04:02.556071+0000","last_became_peered":"2024-03-08T06:04:02.556071+0000","last_unstale":"2024-03-08T06:04:03.511891+0000","last_undegraded":"2024-03-08T06:04:03.511891+0000","last_fullsized":"2024-03-08T06:04:03.511891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617015+0000","last_change":"2024-03-08T06:04:01.556875+0000","last_active":"2024-03-08T06:04:03.617015+0000","last_peered":"2024-03-08T06:04:03.617015+0000","last_clean":"2024-03-08T06:04:03.617015+0000","last_became_active":"2024-03-08T06:04:01.555346+0000","last_became_peered":"2024-03-08T06:04:01.555346+0000","last_unstale":"2024-03-08T06:04:03.617015+0000","last_undegraded":"2024-03-08T06:04:03.617015+0000","last_fullsized":"2024-03-08T06:04:03.617015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.718434+0000","last_change":"2024-03-08T06:04:02.563977+0000","last_active":"2024-03-08T06:04:04.718434+0000","last_peered":"2024-03-08T06:04:04.718434+0000","last_clean":"2024-03-08T06:04:04.718434+0000","last_became_active":"2024-03-08T06:04:02.563653+0000","last_became_peered":"2024-03-08T06:04:02.563653+0000","last_unstale":"2024-03-08T06:04:04.718434+0000","last_undegraded":"2024-03-08T06:04:04.718434+0000","last_fullsized":"2024-03-08T06:04:04.718434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:47:04.734023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.697051+0000","last_change":"2024-03-08T06:04:01.542592+0000","last_active":"2024-03-08T06:04:04.697051+0000","last_peered":"2024-03-08T06:04:04.697051+0000","last_clean":"2024-03-08T06:04:04.697051+0000","last_became_active":"2024-03-08T06:04:01.541567+0000","last_became_peered":"2024-03-08T06:04:01.541567+0000","last_unstale":"2024-03-08T06:04:04.697051+0000","last_undegraded":"2024-03-08T06:04:04.697051+0000","last_fullsized":"2024-03-08T06:04:04.697051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553855+0000","last_change":"2024-03-08T06:04:02.525242+0000","last_active":"2024-03-08T06:04:03.553855+0000","last_peered":"2024-03-08T06:04:03.553855+0000","last_clean":"2024-03-08T06:04:03.553855+0000","last_became_active":"2024-03-08T06:04:02.524598+0000","last_became_peered":"2024-03-08T06:04:02.524598+0000","last_unstale":"2024-03-08T06:04:03.553855+0000","last_undegraded":"2024-03-08T06:04:03.553855+0000","last_fullsized":"2024-03-08T06:04:03.553855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:58:34.852214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516807+0000","last_change":"2024-03-08T06:04:01.557363+0000","last_active":"2024-03-08T06:04:03.516807+0000","last_peered":"2024-03-08T06:04:03.516807+0000","last_clean":"2024-03-08T06:04:03.516807+0000","last_became_active":"2024-03-08T06:04:01.556516+0000","last_became_peered":"2024-03-08T06:04:01.556516+0000","last_unstale":"2024-03-08T06:04:03.516807+0000","last_undegraded":"2024-03-08T06:04:03.516807+0000","last_fullsized":"2024-03-08T06:04:03.516807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118240+0000","last_change":"2024-03-08T06:04:02.554738+0000","last_active":"2024-03-08T06:04:04.118240+0000","last_peered":"2024-03-08T06:04:04.118240+0000","last_clean":"2024-03-08T06:04:04.118240+0000","last_became_active":"2024-03-08T06:04:02.554446+0000","last_became_peered":"2024-03-08T06:04:02.554446+0000","last_unstale":"2024-03-08T06:04:04.118240+0000","last_undegraded":"2024-03-08T06:04:04.118240+0000","last_fullsized":"2024-03-08T06:04:04.118240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:51.718653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553561+0000","last_change":"2024-03-08T06:04:01.543134+0000","last_active":"2024-03-08T06:04:03.553561+0000","last_peered":"2024-03-08T06:04:03.553561+0000","last_clean":"2024-03-08T06:04:03.553561+0000","last_became_active":"2024-03-08T06:04:01.542768+0000","last_became_peered":"2024-03-08T06:04:01.542768+0000","last_unstale":"2024-03-08T06:04:03.553561+0000","last_undegraded":"2024-03-08T06:04:03.553561+0000","last_fullsized":"2024-03-08T06:04:03.553561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517531+0000","last_change":"2024-03-08T06:04:02.549139+0000","last_active":"2024-03-08T06:04:03.517531+0000","last_peered":"2024-03-08T06:04:03.517531+0000","last_clean":"2024-03-08T06:04:03.517531+0000","last_became_active":"2024-03-08T06:04:02.548765+0000","last_became_peered":"2024-03-08T06:04:02.548765+0000","last_unstale":"2024-03-08T06:04:03.517531+0000","last_undegraded":"2024-03-08T06:04:03.517531+0000","last_fullsized":"2024-03-08T06:04:03.517531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:41:43.397781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618664+0000","last_change":"2024-03-08T06:04:01.553438+0000","last_active":"2024-03-08T06:04:03.618664+0000","last_peered":"2024-03-08T06:04:03.618664+0000","last_clean":"2024-03-08T06:04:03.618664+0000","last_became_active":"2024-03-08T06:04:01.553120+0000","last_became_peered":"2024-03-08T06:04:01.553120+0000","last_unstale":"2024-03-08T06:04:03.618664+0000","last_undegraded":"2024-03-08T06:04:03.618664+0000","last_fullsized":"2024-03-08T06:04:03.618664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:05:58.801355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553556+0000","last_change":"2024-03-08T06:04:02.523565+0000","last_active":"2024-03-08T06:04:03.553556+0000","last_peered":"2024-03-08T06:04:03.553556+0000","last_clean":"2024-03-08T06:04:03.553556+0000","last_became_active":"2024-03-08T06:04:02.522603+0000","last_became_peered":"2024-03-08T06:04:02.522603+0000","last_unstale":"2024-03-08T06:04:03.553556+0000","last_undegraded":"2024-03-08T06:04:03.553556+0000","last_fullsized":"2024-03-08T06:04:03.553556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:53:06.699158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.557358+0000","last_change":"2024-03-08T06:04:01.559721+0000","last_active":"2024-03-08T06:04:03.557358+0000","last_peered":"2024-03-08T06:04:03.557358+0000","last_clean":"2024-03-08T06:04:03.557358+0000","last_became_active":"2024-03-08T06:04:01.559454+0000","last_became_peered":"2024-03-08T06:04:01.559454+0000","last_unstale":"2024-03-08T06:04:03.557358+0000","last_undegraded":"2024-03-08T06:04:03.557358+0000","last_fullsized":"2024-03-08T06:04:03.557358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:09:51.288323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512336+0000","last_change":"2024-03-08T06:04:02.555474+0000","last_active":"2024-03-08T06:04:03.512336+0000","last_peered":"2024-03-08T06:04:03.512336+0000","last_clean":"2024-03-08T06:04:03.512336+0000","last_became_active":"2024-03-08T06:04:02.551802+0000","last_became_peered":"2024-03-08T06:04:02.551802+0000","last_unstale":"2024-03-08T06:04:03.512336+0000","last_undegraded":"2024-03-08T06:04:03.512336+0000","last_fullsized":"2024-03-08T06:04:03.512336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:39.146933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559860+0000","last_change":"2024-03-08T06:04:01.559130+0000","last_active":"2024-03-08T06:04:03.559860+0000","last_peered":"2024-03-08T06:04:03.559860+0000","last_clean":"2024-03-08T06:04:03.559860+0000","last_became_active":"2024-03-08T06:04:01.558748+0000","last_became_peered":"2024-03-08T06:04:01.558748+0000","last_unstale":"2024-03-08T06:04:03.559860+0000","last_undegraded":"2024-03-08T06:04:03.559860+0000","last_fullsized":"2024-03-08T06:04:03.559860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719468+0000","last_change":"2024-03-08T06:04:02.544456+0000","last_active":"2024-03-08T06:04:04.719468+0000","last_peered":"2024-03-08T06:04:04.719468+0000","last_clean":"2024-03-08T06:04:04.719468+0000","last_became_active":"2024-03-08T06:04:02.543887+0000","last_became_peered":"2024-03-08T06:04:02.543887+0000","last_unstale":"2024-03-08T06:04:04.719468+0000","last_undegraded":"2024-03-08T06:04:04.719468+0000","last_fullsized":"2024-03-08T06:04:04.719468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:32:00.468984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617019+0000","last_change":"2024-03-08T06:04:01.552997+0000","last_active":"2024-03-08T06:04:03.617019+0000","last_peered":"2024-03-08T06:04:03.617019+0000","last_clean":"2024-03-08T06:04:03.617019+0000","last_became_active":"2024-03-08T06:04:01.552617+0000","last_became_peered":"2024-03-08T06:04:01.552617+0000","last_unstale":"2024-03-08T06:04:03.617019+0000","last_undegraded":"2024-03-08T06:04:03.617019+0000","last_fullsized":"2024-03-08T06:04:03.617019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:15:18.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.118081+0000","last_change":"2024-03-08T06:04:02.554162+0000","last_active":"2024-03-08T06:04:04.118081+0000","last_peered":"2024-03-08T06:04:04.118081+0000","last_clean":"2024-03-08T06:04:04.118081+0000","last_became_active":"2024-03-08T06:04:02.553850+0000","last_became_peered":"2024-03-08T06:04:02.553850+0000","last_unstale":"2024-03-08T06:04:04.118081+0000","last_undegraded":"2024-03-08T06:04:04.118081+0000","last_fullsized":"2024-03-08T06:04:04.118081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:09.964322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553380+0000","last_change":"2024-03-08T06:04:01.547872+0000","last_active":"2024-03-08T06:04:03.553380+0000","last_peered":"2024-03-08T06:04:03.553380+0000","last_clean":"2024-03-08T06:04:03.553380+0000","last_became_active":"2024-03-08T06:04:01.547553+0000","last_became_peered":"2024-03-08T06:04:01.547553+0000","last_unstale":"2024-03-08T06:04:03.553380+0000","last_undegraded":"2024-03-08T06:04:03.553380+0000","last_fullsized":"2024-03-08T06:04:03.553380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511556+0000","last_change":"2024-03-08T06:04:02.546841+0000","last_active":"2024-03-08T06:04:03.511556+0000","last_peered":"2024-03-08T06:04:03.511556+0000","last_clean":"2024-03-08T06:04:03.511556+0000","last_became_active":"2024-03-08T06:04:02.543164+0000","last_became_peered":"2024-03-08T06:04:02.543164+0000","last_unstale":"2024-03-08T06:04:03.511556+0000","last_undegraded":"2024-03-08T06:04:03.511556+0000","last_fullsized":"2024-03-08T06:04:03.511556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.553503+0000","last_change":"2024-03-08T06:04:01.543186+0000","last_active":"2024-03-08T06:04:03.553503+0000","last_peered":"2024-03-08T06:04:03.553503+0000","last_clean":"2024-03-08T06:04:03.553503+0000","last_became_active":"2024-03-08T06:04:01.542185+0000","last_became_peered":"2024-03-08T06:04:01.542185+0000","last_unstale":"2024-03-08T06:04:03.553503+0000","last_undegraded":"2024-03-08T06:04:03.553503+0000","last_fullsized":"2024-03-08T06:04:03.553503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:05:58.665922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.511983+0000","last_change":"2024-03-08T06:04:02.552865+0000","last_active":"2024-03-08T06:04:03.511983+0000","last_peered":"2024-03-08T06:04:03.511983+0000","last_clean":"2024-03-08T06:04:03.511983+0000","last_became_active":"2024-03-08T06:04:02.552563+0000","last_became_peered":"2024-03-08T06:04:02.552563+0000","last_unstale":"2024-03-08T06:04:03.511983+0000","last_undegraded":"2024-03-08T06:04:03.511983+0000","last_fullsized":"2024-03-08T06:04:03.511983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618134+0000","last_change":"2024-03-08T06:04:01.560452+0000","last_active":"2024-03-08T06:04:03.618134+0000","last_peered":"2024-03-08T06:04:03.618134+0000","last_clean":"2024-03-08T06:04:03.618134+0000","last_became_active":"2024-03-08T06:04:01.559883+0000","last_became_peered":"2024-03-08T06:04:01.559883+0000","last_unstale":"2024-03-08T06:04:03.618134+0000","last_undegraded":"2024-03-08T06:04:03.618134+0000","last_fullsized":"2024-03-08T06:04:03.618134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:50.471516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.559002+0000","last_change":"2024-03-08T06:04:02.562767+0000","last_active":"2024-03-08T06:04:03.559002+0000","last_peered":"2024-03-08T06:04:03.559002+0000","last_clean":"2024-03-08T06:04:03.559002+0000","last_became_active":"2024-03-08T06:04:02.562425+0000","last_became_peered":"2024-03-08T06:04:02.562425+0000","last_unstale":"2024-03-08T06:04:03.559002+0000","last_undegraded":"2024-03-08T06:04:03.559002+0000","last_fullsized":"2024-03-08T06:04:03.559002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:15:51.623231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.517395+0000","last_change":"2024-03-08T06:04:02.559515+0000","last_active":"2024-03-08T06:04:03.517395+0000","last_peered":"2024-03-08T06:04:03.517395+0000","last_clean":"2024-03-08T06:04:03.517395+0000","last_became_active":"2024-03-08T06:04:02.559082+0000","last_became_peered":"2024-03-08T06:04:02.559082+0000","last_unstale":"2024-03-08T06:04:03.517395+0000","last_undegraded":"2024-03-08T06:04:03.517395+0000","last_fullsized":"2024-03-08T06:04:03.517395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:42:12.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.617836+0000","last_change":"2024-03-08T06:04:01.560587+0000","last_active":"2024-03-08T06:04:03.617836+0000","last_peered":"2024-03-08T06:04:03.617836+0000","last_clean":"2024-03-08T06:04:03.617836+0000","last_became_active":"2024-03-08T06:04:01.560275+0000","last_became_peered":"2024-03-08T06:04:01.560275+0000","last_unstale":"2024-03-08T06:04:03.617836+0000","last_undegraded":"2024-03-08T06:04:03.617836+0000","last_fullsized":"2024-03-08T06:04:03.617836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:33:08.717185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.516385+0000","last_change":"2024-03-08T06:04:02.559339+0000","last_active":"2024-03-08T06:04:03.516385+0000","last_peered":"2024-03-08T06:04:03.516385+0000","last_clean":"2024-03-08T06:04:03.516385+0000","last_became_active":"2024-03-08T06:04:02.559023+0000","last_became_peered":"2024-03-08T06:04:02.559023+0000","last_unstale":"2024-03-08T06:04:03.516385+0000","last_undegraded":"2024-03-08T06:04:03.516385+0000","last_fullsized":"2024-03-08T06:04:03.516385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:24:30.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.618371+0000","last_change":"2024-03-08T06:04:01.560256+0000","last_active":"2024-03-08T06:04:03.618371+0000","last_peered":"2024-03-08T06:04:03.618371+0000","last_clean":"2024-03-08T06:04:03.618371+0000","last_became_active":"2024-03-08T06:04:01.559917+0000","last_became_peered":"2024-03-08T06:04:01.559917+0000","last_unstale":"2024-03-08T06:04:03.618371+0000","last_undegraded":"2024-03-08T06:04:03.618371+0000","last_fullsized":"2024-03-08T06:04:03.618371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:38:55.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719619+0000","last_change":"2024-03-08T06:04:02.551405+0000","last_active":"2024-03-08T06:04:04.719619+0000","last_peered":"2024-03-08T06:04:04.719619+0000","last_clean":"2024-03-08T06:04:04.719619+0000","last_became_active":"2024-03-08T06:04:02.550283+0000","last_became_peered":"2024-03-08T06:04:02.550283+0000","last_unstale":"2024-03-08T06:04:04.719619+0000","last_undegraded":"2024-03-08T06:04:04.719619+0000","last_fullsized":"2024-03-08T06:04:04.719619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:16:16.623660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512142+0000","last_change":"2024-03-08T06:04:01.540725+0000","last_active":"2024-03-08T06:04:03.512142+0000","last_peered":"2024-03-08T06:04:03.512142+0000","last_clean":"2024-03-08T06:04:03.512142+0000","last_became_active":"2024-03-08T06:04:01.540349+0000","last_became_peered":"2024-03-08T06:04:01.540349+0000","last_unstale":"2024-03-08T06:04:03.512142+0000","last_undegraded":"2024-03-08T06:04:03.512142+0000","last_fullsized":"2024-03-08T06:04:03.512142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:57:13.857302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:03.512201+0000","last_change":"2024-03-08T06:04:02.554282+0000","last_active":"2024-03-08T06:04:03.512201+0000","last_peered":"2024-03-08T06:04:03.512201+0000","last_clean":"2024-03-08T06:04:03.512201+0000","last_became_active":"2024-03-08T06:04:02.551101+0000","last_became_peered":"2024-03-08T06:04:02.551101+0000","last_unstale":"2024-03-08T06:04:03.512201+0000","last_undegraded":"2024-03-08T06:04:03.512201+0000","last_fullsized":"2024-03-08T06:04:03.512201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:01.497931+0000","last_clean_scrub_stamp":"2024-03-08T06:04:01.497931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:10.566436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-08T06:04:04.719812+0000","last_change":"2024-03-08T06:04:01.555646+0000","last_active":"2024-03-08T06:04:04.719812+0000","last_peered":"2024-03-08T06:04:04.719812+0000","last_clean":"2024-03-08T06:04:04.719812+0000","last_became_active":"2024-03-08T06:04:01.555404+0000","last_became_peered":"2024-03-08T06:04:01.555404+0000","last_unstale":"2024-03-08T06:04:04.719812+0000","last_undegraded":"2024-03-08T06:04:04.719812+0000","last_fullsized":"2024-03-08T06:04:04.719812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:04:00.250795+0000","last_clean_scrub_stamp":"2024-03-08T06:04:00.250795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:52:36.740849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:04:11.311 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-08T06:04:11.311 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-08T06:04:11.311 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-08T06:04:11.311 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.567+0000 7f749d400640 1 Processor -- start 2024-03-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.567+0000 7f749d400640 1 -- start start 2024-03-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.569+0000 7f749d400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7498145400 0x7f74980cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.569+0000 7f749d400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f749815f6e0 0x7f74980cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.569+0000 7f749d400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74980ccd30 con 0x7f74980ce140 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.569+0000 7f749d400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74980ce590 con 0x7f7498145400 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.569+0000 7f749d400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74980cd940 con 0x7f749815f6e0 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.569+0000 7f7496800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7498145400 0x7f74980cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.569+0000 7f7496800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7498145400 0x7f74980cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45800/0 (socket says 172.21.15.40:45800) 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.569+0000 7f7496800640 1 -- 172.21.15.40:0/976207399 learned_addr learned my addr 172.21.15.40:0/976207399 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.569+0000 7f7495e00640 1 --2- 172.21.15.40:0/976207399 >> v2:172.21.15.53:3300/0 conn(0x7f749815f6e0 0x7f74980cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.570+0000 7f7496800640 1 -- 172.21.15.40:0/976207399 >> v2:172.21.15.53:3300/0 conn(0x7f749815f6e0 msgr2=0x7f74980cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.570+0000 7f7496800640 1 --2- 172.21.15.40:0/976207399 >> v2:172.21.15.53:3300/0 conn(0x7f749815f6e0 0x7f74980cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.570+0000 7f7496800640 1 -- 172.21.15.40:0/976207399 >> v1:172.21.15.53:6789/0 conn(0x7f74980ce140 legacy=0x7f749815c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.570+0000 7f7496800640 1 -- 172.21.15.40:0/976207399 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74980cd010 con 0x7f7498145400 2024-03-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.571+0000 7f7496800640 1 --2- 172.21.15.40:0/976207399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498145400 0x7f74980cc3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f74980b1690 tx=0x7f748802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf0ac39dbb73aeb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.571+0000 7f7495400640 1 -- 172.21.15.40:0/976207399 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488034070 con 0x7f7498145400 2024-03-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.571+0000 7f7495400640 1 -- 172.21.15.40:0/976207399 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7488030860 con 0x7f7498145400 2024-03-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.571+0000 7f7495400640 1 -- 172.21.15.40:0/976207399 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488030c30 con 0x7f7498145400 2024-03-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.572+0000 7f749d400640 1 -- 172.21.15.40:0/976207399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498145400 msgr2=0x7f74980cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.572+0000 7f749d400640 1 --2- 172.21.15.40:0/976207399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498145400 0x7f74980cc3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f74980b1690 tx=0x7f748802b9a0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.572+0000 7f749d400640 1 -- 172.21.15.40:0/976207399 shutdown_connections 2024-03-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.572+0000 7f749d400640 1 --2- 172.21.15.40:0/976207399 >> v2:172.21.15.53:3300/0 conn(0x7f749815f6e0 0x7f74980cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.572+0000 7f749d400640 1 --2- 172.21.15.40:0/976207399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498145400 0x7f74980cc3c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.572+0000 7f749d400640 1 -- 172.21.15.40:0/976207399 >> 172.21.15.40:0/976207399 conn(0x7f74980b16e0 msgr2=0x7f74980b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.572+0000 7f749d400640 1 -- 172.21.15.40:0/976207399 shutdown_connections 2024-03-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.572+0000 7f749d400640 1 -- 172.21.15.40:0/976207399 wait complete. 2024-03-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.574+0000 7f749d400640 1 Processor -- start 2024-03-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.574+0000 7f749d400640 1 -- start start 2024-03-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f749d400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74980ce140 0x7f749815be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f749d400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7498145400 0x7f749813e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f749d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f749815f6e0 0x7f74981f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f749d400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980cc900 con 0x7f74980ce140 2024-03-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f749d400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980cd5d0 con 0x7f749815f6e0 2024-03-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f749d400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74980cca80 con 0x7f7498145400 2024-03-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f7496800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74980ce140 0x7f749815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f7496800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74980ce140 0x7f749815be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45806/0 (socket says 172.21.15.40:45806) 2024-03-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f7496800640 1 -- 172.21.15.40:0/966630631 learned_addr learned my addr 172.21.15.40:0/966630631 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f7496800640 1 -- 172.21.15.40:0/966630631 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7498145400 msgr2=0x7f749813e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f7496800640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7498145400 0x7f749813e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f7496800640 1 -- 172.21.15.40:0/966630631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f749815f6e0 msgr2=0x7f74981f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f7497200640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f749815f6e0 0x7f74981f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f7496800640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f749815f6e0 0x7f74981f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.575+0000 7f7496800640 1 -- 172.21.15.40:0/966630631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749815c3b0 con 0x7f74980ce140 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.576+0000 7f7496800640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74980ce140 0x7f749815be70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f748802b9e0 tx=0x7f7488004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.576+0000 7f748ea00640 1 -- 172.21.15.40:0/966630631 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488048070 con 0x7f74980ce140 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.576+0000 7f748ea00640 1 -- 172.21.15.40:0/966630631 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7488030660 con 0x7f74980ce140 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.576+0000 7f748ea00640 1 -- 172.21.15.40:0/966630631 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488034040 con 0x7f74980ce140 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.576+0000 7f749d400640 1 -- 172.21.15.40:0/966630631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749813f2a0 con 0x7f74980ce140 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.576+0000 7f749d400640 1 -- 172.21.15.40:0/966630631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749813f860 con 0x7f74980ce140 2024-03-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.577+0000 7f749d400640 1 -- 172.21.15.40:0/966630631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7498067cb0 con 0x7f74980ce140 2024-03-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.579+0000 7f748ea00640 1 -- 172.21.15.40:0/966630631 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7488037070 con 0x7f74980ce140 2024-03-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.579+0000 7f748ea00640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f747806b500 0x7f747806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.579+0000 7f748ea00640 1 -- 172.21.15.40:0/966630631 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74880abf30 con 0x7f74980ce140 2024-03-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.580+0000 7f7495e00640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f747806b500 0x7f747806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.580+0000 7f7495e00640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f747806b500 0x7f747806d9c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f749815f050 tx=0x7f7484008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.588+0000 7f748ea00640 1 -- 172.21.15.40:0/966630631 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f748807ada0 con 0x7f74980ce140 2024-03-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.816+0000 7f749d400640 1 -- 172.21.15.40:0/966630631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f749806dc00 con 0x7f74980ce140 2024-03-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.818+0000 7f748ea00640 1 -- 172.21.15.40:0/966630631 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f748807a6a0 con 0x7f74980ce140 2024-03-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 -- 172.21.15.40:0/966630631 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f747806b500 msgr2=0x7f747806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f747806b500 0x7f747806d9c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f749815f050 tx=0x7f7484008040 comp rx=0 tx=0).stop 2024-03-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 -- 172.21.15.40:0/966630631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74980ce140 msgr2=0x7f749815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74980ce140 0x7f749815be70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f748802b9e0 tx=0x7f7488004730 comp rx=0 tx=0).stop 2024-03-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 -- 172.21.15.40:0/966630631 shutdown_connections 2024-03-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f747806b500 0x7f747806d9c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f749815f6e0 0x7f74981f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7498145400 0x7f749813e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 --2- 172.21.15.40:0/966630631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74980ce140 0x7f749815be70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 -- 172.21.15.40:0/966630631 >> 172.21.15.40:0/966630631 conn(0x7f74980b16e0 msgr2=0x7f74980af650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 -- 172.21.15.40:0/966630631 shutdown_connections 2024-03-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:11.820+0000 7f749d400640 1 -- 172.21.15.40:0/966630631 wait complete. 2024-03-08T06:04:11.837 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-08T06:04:11.837 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.042+0000 7fa7bf200640 1 Processor -- start 2024-03-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.043+0000 7fa7bf200640 1 -- start start 2024-03-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.044+0000 7fa7bf200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c015df20 0x7fa7c00ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.044+0000 7fa7bf200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa7c00b6bd0 0x7fa7c0162400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.044+0000 7fa7bf200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c015e300 con 0x7fa7c00d0aa0 2024-03-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.044+0000 7fa7bf200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c00d3990 con 0x7fa7c00b6bd0 2024-03-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.044+0000 7fa7bf200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c0162940 con 0x7fa7c015df20 2024-03-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.045+0000 7fa7be800640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa7c00d0aa0 0x7fa7c00b5590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60968/0 (socket says 172.21.15.40:60968) 2024-03-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.045+0000 7fa7be800640 1 -- 172.21.15.40:0/1133647466 learned_addr learned my addr 172.21.15.40:0/1133647466 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.045+0000 7fa7bde00640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c015df20 0x7fa7c00ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.045+0000 7fa7bd400640 1 --2- 172.21.15.40:0/1133647466 >> v2:172.21.15.53:3300/0 conn(0x7fa7c00b6bd0 0x7fa7c0162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.045+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283811548 0 0) 0x7fa7c015e300 con 0x7fa7c00d0aa0 2024-03-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.045+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a4003520 con 0x7fa7c00d0aa0 2024-03-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1093651903 0 0) 0x7fa7a4003520 con 0x7fa7c00d0aa0 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c015df20 msgr2=0x7fa7c00ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c015df20 0x7fa7c00ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 >> v2:172.21.15.53:3300/0 conn(0x7fa7c00b6bd0 msgr2=0x7fa7c0162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 --2- 172.21.15.40:0/1133647466 >> v2:172.21.15.53:3300/0 conn(0x7fa7c00b6bd0 0x7fa7c0162400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c00cf030 con 0x7fa7c00d0aa0 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fa7b8001670 con 0x7fa7c00d0aa0 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bd400640 1 --2- 172.21.15.40:0/1133647466 >> v2:172.21.15.53:3300/0 conn(0x7fa7c00b6bd0 0x7fa7c0162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bde00640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c015df20 0x7fa7c00ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 >> v1:172.21.15.40:6789/0 conn(0x7fa7c00d0aa0 legacy=0x7fa7c00b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c015df20 0x7fa7a40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 0x7fa7a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7a4004d40 0x7fa7a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00d3990 con 0x7fa7a4003ef0 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c0162940 con 0x7fa7a4004d40 2024-03-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a400b9c0 con 0x7fa7c015df20 2024-03-08T06:04:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bd400640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 0x7fa7a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bde00640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c015df20 0x7fa7a40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7be800640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7a4004d40 0x7fa7a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bd400640 1 -- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c015df20 msgr2=0x7fa7a40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.046+0000 7fa7bd400640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c015df20 0x7fa7a40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.047+0000 7fa7bd400640 1 -- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7a4004d40 msgr2=0x7fa7a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.047+0000 7fa7bd400640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7a4004d40 0x7fa7a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.047+0000 7fa7bd400640 1 -- 172.21.15.40:0/1133647466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a4003520 con 0x7fa7a4003ef0 2024-03-08T06:04:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.047+0000 7fa7bde00640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c015df20 0x7fa7a40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.047+0000 7fa7be800640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7a4004d40 0x7fa7a40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.048+0000 7fa7bd400640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 0x7fa7a4004800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00e380 tx=0x7fa7a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4ec5f44814c9ac server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.048+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac039070 con 0x7fa7a4003ef0 2024-03-08T06:04:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.048+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac02cde0 con 0x7fa7a4003ef0 2024-03-08T06:04:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.048+0000 7fa7bca00640 1 -- 172.21.15.40:0/1133647466 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac009de0 con 0x7fa7a4003ef0 2024-03-08T06:04:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.049+0000 7fa7bf200640 1 -- 172.21.15.40:0/1133647466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 msgr2=0x7fa7a4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.049+0000 7fa7bf200640 1 --2- 172.21.15.40:0/1133647466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 0x7fa7a4004800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00e380 tx=0x7fa7a4005830 comp rx=0 tx=0).stop 2024-03-08T06:04:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.049+0000 7fa7be800640 1 -- 172.21.15.40:0/1133647466 reap_dead start 2024-03-08T06:04:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.049+0000 7fa7bf200640 1 -- 172.21.15.40:0/1133647466 shutdown_connections 2024-03-08T06:04:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.049+0000 7fa7bf200640 1 -- 172.21.15.40:0/1133647466 >> 172.21.15.40:0/1133647466 conn(0x7fa7c006c740 msgr2=0x7fa7c006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.049+0000 7fa7bf200640 1 -- 172.21.15.40:0/1133647466 shutdown_connections 2024-03-08T06:04:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.049+0000 7fa7bf200640 1 -- 172.21.15.40:0/1133647466 wait complete. 2024-03-08T06:04:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.051+0000 7fa7bf200640 1 Processor -- start 2024-03-08T06:04:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.051+0000 7fa7bf200640 1 -- start start 2024-03-08T06:04:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.052+0000 7fa7bf200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 0x7fa7c0160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.052+0000 7fa7bf200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c01f6610 0x7fa7c00cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.052+0000 7fa7bf200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c00b8a20 0x7fa7c00b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.052+0000 7fa7bf200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00d2180 con 0x7fa7a4003ef0 2024-03-08T06:04:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.052+0000 7fa7bf200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00d1ea0 con 0x7fa7c00b8a20 2024-03-08T06:04:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.052+0000 7fa7bf200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00d2460 con 0x7fa7c01f6610 2024-03-08T06:04:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.052+0000 7fa7bde00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 0x7fa7c0160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.052+0000 7fa7bde00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 0x7fa7c0160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45826/0 (socket says 172.21.15.40:45826) 2024-03-08T06:04:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.052+0000 7fa7bde00640 1 -- 172.21.15.40:0/1102251231 learned_addr learned my addr 172.21.15.40:0/1102251231 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.053+0000 7fa7bde00640 1 -- 172.21.15.40:0/1102251231 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c01f6610 msgr2=0x7fa7c00cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.053+0000 7fa7bde00640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c01f6610 0x7fa7c00cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.053+0000 7fa7bde00640 1 -- 172.21.15.40:0/1102251231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c00b8a20 msgr2=0x7fa7c00b8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.053+0000 7fa7bde00640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c00b8a20 0x7fa7c00b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.053+0000 7fa7bde00640 1 -- 172.21.15.40:0/1102251231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c00cb350 con 0x7fa7a4003ef0 2024-03-08T06:04:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.053+0000 7fa7bde00640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 0x7fa7c0160900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0010810 tx=0x7fa7b0010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.053+0000 7fa7b6000640 1 -- 172.21.15.40:0/1102251231 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b000f040 con 0x7fa7a4003ef0 2024-03-08T06:04:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.053+0000 7fa7b6000640 1 -- 172.21.15.40:0/1102251231 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b000eb20 con 0x7fa7a4003ef0 2024-03-08T06:04:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.053+0000 7fa7b6000640 1 -- 172.21.15.40:0/1102251231 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b001c400 con 0x7fa7a4003ef0 2024-03-08T06:04:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.053+0000 7fa7bf200640 1 -- 172.21.15.40:0/1102251231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c00cb630 con 0x7fa7a4003ef0 2024-03-08T06:04:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.054+0000 7fa7bf200640 1 -- 172.21.15.40:0/1102251231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c00cbbf0 con 0x7fa7a4003ef0 2024-03-08T06:04:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.055+0000 7fa7bf200640 1 -- 172.21.15.40:0/1102251231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c0075100 con 0x7fa7a4003ef0 2024-03-08T06:04:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.056+0000 7fa7b6000640 1 -- 172.21.15.40:0/1102251231 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7b001c5a0 con 0x7fa7a4003ef0 2024-03-08T06:04:12.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.056+0000 7fa7b6000640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7a006b500 0x7fa7a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.057+0000 7fa7b6000640 1 -- 172.21.15.40:0/1102251231 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7b008a760 con 0x7fa7a4003ef0 2024-03-08T06:04:12.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.057+0000 7fa7bd400640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7a006b500 0x7fa7a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.058+0000 7fa7bd400640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7a006b500 0x7fa7a006d9c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac036000 tx=0x7fa7ac02e070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:12.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.061+0000 7fa7b6000640 1 -- 172.21.15.40:0/1102251231 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7b0018030 con 0x7fa7a4003ef0 2024-03-08T06:04:12.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.264+0000 7fa7bf200640 1 -- 172.21.15.40:0/1102251231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7c00d16e0 con 0x7fa7a4003ef0 2024-03-08T06:04:12.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.267+0000 7fa7b6000640 1 -- 172.21.15.40:0/1102251231 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7fa7b0058f80 con 0x7fa7a4003ef0 2024-03-08T06:04:12.268 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:12.268 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:12.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.270+0000 7fa7bf200640 1 -- 172.21.15.40:0/1102251231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7a006b500 msgr2=0x7fa7a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.270+0000 7fa7bf200640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7a006b500 0x7fa7a006d9c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac036000 tx=0x7fa7ac02e070 comp rx=0 tx=0).stop 2024-03-08T06:04:12.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.270+0000 7fa7bf200640 1 -- 172.21.15.40:0/1102251231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 msgr2=0x7fa7c0160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.270+0000 7fa7bf200640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 0x7fa7c0160900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0010810 tx=0x7fa7b0010cd0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.271+0000 7fa7bf200640 1 -- 172.21.15.40:0/1102251231 shutdown_connections 2024-03-08T06:04:12.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.271+0000 7fa7bf200640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7a006b500 0x7fa7a006d9c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.271+0000 7fa7bf200640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c00b8a20 0x7fa7c00b8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.271+0000 7fa7bf200640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7c01f6610 0x7fa7c00cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.271+0000 7fa7bf200640 1 --2- 172.21.15.40:0/1102251231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7a4003ef0 0x7fa7c0160900 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.271+0000 7fa7bf200640 1 -- 172.21.15.40:0/1102251231 >> 172.21.15.40:0/1102251231 conn(0x7fa7c006c740 msgr2=0x7fa7c0074410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:12.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.271+0000 7fa7bf200640 1 -- 172.21.15.40:0/1102251231 shutdown_connections 2024-03-08T06:04:12.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.271+0000 7fa7bf200640 1 -- 172.21.15.40:0/1102251231 wait complete. 2024-03-08T06:04:12.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:12.291 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.515+0000 7fd5eb200640 1 Processor -- start 2024-03-08T06:04:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.515+0000 7fd5eb200640 1 -- start start 2024-03-08T06:04:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.517+0000 7fd5eb200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd5e4144240 0x7fd5e40cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.517+0000 7fd5eb200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5e40b42c0 0x7fd5e406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.517+0000 7fd5eb200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e40cd3e0 con 0x7fd5e40ccef0 2024-03-08T06:04:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.517+0000 7fd5eb200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e40b3e30 con 0x7fd5e4144240 2024-03-08T06:04:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.517+0000 7fd5eb200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40cd6c0 con 0x7fd5e40b42c0 2024-03-08T06:04:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.517+0000 7fd5e9e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd5e4144240 0x7fd5e40cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.517+0000 7fd5e9e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd5e4144240 0x7fd5e40cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:49814/0 (socket says 172.21.15.40:49814) 2024-03-08T06:04:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.517+0000 7fd5e9e00640 1 -- 172.21.15.40:0/338293668 learned_addr learned my addr 172.21.15.40:0/338293668 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.517+0000 7fd5e9400640 1 --2- 172.21.15.40:0/338293668 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5e40b42c0 0x7fd5e406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.518+0000 7fd5e9e00640 1 -- 172.21.15.40:0/338293668 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5e40b42c0 msgr2=0x7fd5e406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.518+0000 7fd5e9e00640 1 --2- 172.21.15.40:0/338293668 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5e40b42c0 0x7fd5e406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.518+0000 7fd5e9e00640 1 -- 172.21.15.40:0/338293668 >> v1:172.21.15.53:6789/0 conn(0x7fd5e40ccef0 legacy=0x7fd5e40716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.518+0000 7fd5e9e00640 1 -- 172.21.15.40:0/338293668 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e40b37e0 con 0x7fd5e4144240 2024-03-08T06:04:12.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.519+0000 7fd5e9e00640 1 --2- 172.21.15.40:0/338293668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e4144240 0x7fd5e40cc9b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009870 tx=0x7fd5d002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f785c2c57f592d9f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:12.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.520+0000 7fd5e8a00640 1 -- 172.21.15.40:0/338293668 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0034070 con 0x7fd5e4144240 2024-03-08T06:04:12.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.520+0000 7fd5e8a00640 1 -- 172.21.15.40:0/338293668 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d0030840 con 0x7fd5e4144240 2024-03-08T06:04:12.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.520+0000 7fd5e8a00640 1 -- 172.21.15.40:0/338293668 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0030c10 con 0x7fd5e4144240 2024-03-08T06:04:12.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.520+0000 7fd5eb200640 1 -- 172.21.15.40:0/338293668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e4144240 msgr2=0x7fd5e40cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.520+0000 7fd5eb200640 1 --2- 172.21.15.40:0/338293668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e4144240 0x7fd5e40cc9b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009870 tx=0x7fd5d002b980 comp rx=0 tx=0).stop 2024-03-08T06:04:12.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.520+0000 7fd5eb200640 1 -- 172.21.15.40:0/338293668 shutdown_connections 2024-03-08T06:04:12.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.520+0000 7fd5eb200640 1 --2- 172.21.15.40:0/338293668 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5e40b42c0 0x7fd5e406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.520+0000 7fd5eb200640 1 --2- 172.21.15.40:0/338293668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e4144240 0x7fd5e40cc9b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.520+0000 7fd5eb200640 1 -- 172.21.15.40:0/338293668 >> 172.21.15.40:0/338293668 conn(0x7fd5e40b1850 msgr2=0x7fd5e40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:12.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.521+0000 7fd5eb200640 1 -- 172.21.15.40:0/338293668 shutdown_connections 2024-03-08T06:04:12.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.521+0000 7fd5eb200640 1 -- 172.21.15.40:0/338293668 wait complete. 2024-03-08T06:04:12.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.522+0000 7fd5eb200640 1 Processor -- start 2024-03-08T06:04:12.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.523+0000 7fd5eb200640 1 -- start start 2024-03-08T06:04:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.523+0000 7fd5eb200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e40ccef0 0x7fd5e4138790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.523+0000 7fd5eb200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5e4144240 0x7fd5e415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5eb200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e41333b0 0x7fd5e406d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5eb200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40b4b30 con 0x7fd5e41333b0 2024-03-08T06:04:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5eb200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e4144620 con 0x7fd5e40ccef0 2024-03-08T06:04:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5eb200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40b4cb0 con 0x7fd5e4144240 2024-03-08T06:04:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5ea800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e41333b0 0x7fd5e406d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5ea800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e41333b0 0x7fd5e406d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45838/0 (socket says 172.21.15.40:45838) 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5ea800640 1 -- 172.21.15.40:0/3064899608 learned_addr learned my addr 172.21.15.40:0/3064899608 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5e9e00640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e40ccef0 0x7fd5e4138790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5e9400640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5e4144240 0x7fd5e415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5ea800640 1 -- 172.21.15.40:0/3064899608 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5e4144240 msgr2=0x7fd5e415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5ea800640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5e4144240 0x7fd5e415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5ea800640 1 -- 172.21.15.40:0/3064899608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e40ccef0 msgr2=0x7fd5e4138790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5ea800640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e40ccef0 0x7fd5e4138790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5ea800640 1 -- 172.21.15.40:0/3064899608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e4133be0 con 0x7fd5e41333b0 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.524+0000 7fd5e9e00640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e40ccef0 0x7fd5e4138790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.525+0000 7fd5ea800640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e41333b0 0x7fd5e406d730 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c790 tx=0x7fd5e000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.525+0000 7fd5de000640 1 -- 172.21.15.40:0/3064899608 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0016020 con 0x7fd5e41333b0 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.525+0000 7fd5de000640 1 -- 172.21.15.40:0/3064899608 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0006d50 con 0x7fd5e41333b0 2024-03-08T06:04:12.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.525+0000 7fd5de000640 1 -- 172.21.15.40:0/3064899608 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0003bb0 con 0x7fd5e41333b0 2024-03-08T06:04:12.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.525+0000 7fd5eb200640 1 -- 172.21.15.40:0/3064899608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e4133ec0 con 0x7fd5e41333b0 2024-03-08T06:04:12.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.525+0000 7fd5eb200640 1 -- 172.21.15.40:0/3064899608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e4134480 con 0x7fd5e41333b0 2024-03-08T06:04:12.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.526+0000 7fd5eb200640 1 -- 172.21.15.40:0/3064899608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e4075090 con 0x7fd5e41333b0 2024-03-08T06:04:12.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.527+0000 7fd5de000640 1 -- 172.21.15.40:0/3064899608 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5e000a7f0 con 0x7fd5e41333b0 2024-03-08T06:04:12.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.528+0000 7fd5de000640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5c406b500 0x7fd5c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.528+0000 7fd5de000640 1 -- 172.21.15.40:0/3064899608 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5e0089d60 con 0x7fd5e41333b0 2024-03-08T06:04:12.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.528+0000 7fd5e9e00640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5c406b500 0x7fd5c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.529+0000 7fd5e9e00640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5c406b500 0x7fd5c406d9c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0036000 tx=0x7fd5d003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:12.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.533+0000 7fd5de000640 1 -- 172.21.15.40:0/3064899608 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5e0020020 con 0x7fd5e41333b0 2024-03-08T06:04:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.722+0000 7fd5eb200640 1 -- 172.21.15.40:0/3064899608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5e406f160 con 0x7fd5e41333b0 2024-03-08T06:04:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.723+0000 7fd5de000640 1 -- 172.21.15.40:0/3064899608 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5e0058c70 con 0x7fd5e41333b0 2024-03-08T06:04:12.724 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:12.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.725+0000 7fd5eb200640 1 -- 172.21.15.40:0/3064899608 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5c406b500 msgr2=0x7fd5c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.725+0000 7fd5eb200640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5c406b500 0x7fd5c406d9c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0036000 tx=0x7fd5d003b040 comp rx=0 tx=0).stop 2024-03-08T06:04:12.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.726+0000 7fd5eb200640 1 -- 172.21.15.40:0/3064899608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e41333b0 msgr2=0x7fd5e406d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.726+0000 7fd5eb200640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e41333b0 0x7fd5e406d730 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c790 tx=0x7fd5e000cc50 comp rx=0 tx=0).stop 2024-03-08T06:04:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.726+0000 7fd5eb200640 1 -- 172.21.15.40:0/3064899608 shutdown_connections 2024-03-08T06:04:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.726+0000 7fd5eb200640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5c406b500 0x7fd5c406d9c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.726+0000 7fd5eb200640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e41333b0 0x7fd5e406d730 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.726+0000 7fd5eb200640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5e4144240 0x7fd5e415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.726+0000 7fd5eb200640 1 --2- 172.21.15.40:0/3064899608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e40ccef0 0x7fd5e4138790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.726+0000 7fd5eb200640 1 -- 172.21.15.40:0/3064899608 >> 172.21.15.40:0/3064899608 conn(0x7fd5e40b1850 msgr2=0x7fd5e406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.726+0000 7fd5eb200640 1 -- 172.21.15.40:0/3064899608 shutdown_connections 2024-03-08T06:04:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.726+0000 7fd5eb200640 1 -- 172.21.15.40:0/3064899608 wait complete. 2024-03-08T06:04:12.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:12.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.964+0000 7f87dbe00640 1 Processor -- start 2024-03-08T06:04:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.965+0000 7f87dbe00640 1 -- start start 2024-03-08T06:04:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.966+0000 7f87dbe00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f87d41604f0 0x7f87d40cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.966+0000 7f87dbe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87d4071950 0x7f87d4071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.966+0000 7f87dbe00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d40cd5a0 con 0x7f87d41455b0 2024-03-08T06:04:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.966+0000 7f87dbe00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d40cdc20 con 0x7f87d4071950 2024-03-08T06:04:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.966+0000 7f87dbe00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d40722d0 con 0x7f87d41604f0 2024-03-08T06:04:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.966+0000 7f87d9e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87d4071950 0x7f87d4071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.967+0000 7f87d9e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87d4071950 0x7f87d4071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45854/0 (socket says 172.21.15.40:45854) 2024-03-08T06:04:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.967+0000 7f87d9e00640 1 -- 172.21.15.40:0/2552826037 learned_addr learned my addr 172.21.15.40:0/2552826037 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.967+0000 7f87d8a00640 1 --2- 172.21.15.40:0/2552826037 >> v2:172.21.15.53:3300/0 conn(0x7f87d41604f0 0x7f87d40cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.967+0000 7f87d9e00640 1 -- 172.21.15.40:0/2552826037 >> v2:172.21.15.53:3300/0 conn(0x7f87d41604f0 msgr2=0x7f87d40cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.967+0000 7f87d9e00640 1 --2- 172.21.15.40:0/2552826037 >> v2:172.21.15.53:3300/0 conn(0x7f87d41604f0 0x7f87d40cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.967+0000 7f87d9e00640 1 -- 172.21.15.40:0/2552826037 >> v1:172.21.15.53:6789/0 conn(0x7f87d41455b0 legacy=0x7f87d415c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.967+0000 7f87d9e00640 1 -- 172.21.15.40:0/2552826037 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d40ccfe0 con 0x7f87d4071950 2024-03-08T06:04:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.968+0000 7f87d9e00640 1 --2- 172.21.15.40:0/2552826037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87d4071950 0x7f87d4071d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f87d415f730 tx=0x7f87cc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b94759e1130d92 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.968+0000 7f87d3e00640 1 -- 172.21.15.40:0/2552826037 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc02ca80 con 0x7f87d4071950 2024-03-08T06:04:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.969+0000 7f87d3e00640 1 -- 172.21.15.40:0/2552826037 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc02cc20 con 0x7f87d4071950 2024-03-08T06:04:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.969+0000 7f87d3e00640 1 -- 172.21.15.40:0/2552826037 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0343f0 con 0x7f87d4071950 2024-03-08T06:04:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.969+0000 7f87dbe00640 1 -- 172.21.15.40:0/2552826037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87d4071950 msgr2=0x7f87d4071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.969+0000 7f87dbe00640 1 --2- 172.21.15.40:0/2552826037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87d4071950 0x7f87d4071d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f87d415f730 tx=0x7f87cc02bee0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.969+0000 7f87dbe00640 1 -- 172.21.15.40:0/2552826037 shutdown_connections 2024-03-08T06:04:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.969+0000 7f87dbe00640 1 --2- 172.21.15.40:0/2552826037 >> v2:172.21.15.53:3300/0 conn(0x7f87d41604f0 0x7f87d40cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.969+0000 7f87dbe00640 1 --2- 172.21.15.40:0/2552826037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87d4071950 0x7f87d4071d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.969+0000 7f87dbe00640 1 -- 172.21.15.40:0/2552826037 >> 172.21.15.40:0/2552826037 conn(0x7f87d406ae30 msgr2=0x7f87d406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.969+0000 7f87dbe00640 1 -- 172.21.15.40:0/2552826037 shutdown_connections 2024-03-08T06:04:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.969+0000 7f87dbe00640 1 -- 172.21.15.40:0/2552826037 wait complete. 2024-03-08T06:04:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.971+0000 7f87dbe00640 1 Processor -- start 2024-03-08T06:04:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.971+0000 7f87dbe00640 1 -- start start 2024-03-08T06:04:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.972+0000 7f87dbe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87d4071950 0x7f87d41f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.972+0000 7f87dbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87d41455b0 0x7f87d413ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.972+0000 7f87dbe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87d41604f0 0x7f87d413f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.972+0000 7f87dbe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4145990 con 0x7f87d4071950 2024-03-08T06:04:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.972+0000 7f87dbe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40cd8b0 con 0x7f87d41455b0 2024-03-08T06:04:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.972+0000 7f87dbe00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40ccbd0 con 0x7f87d41604f0 2024-03-08T06:04:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.973+0000 7f87d9400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87d4071950 0x7f87d41f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.973+0000 7f87d9400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87d4071950 0x7f87d41f1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45858/0 (socket says 172.21.15.40:45858) 2024-03-08T06:04:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.973+0000 7f87d9400640 1 -- 172.21.15.40:0/1372674973 learned_addr learned my addr 172.21.15.40:0/1372674973 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.973+0000 7f87d8a00640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87d41455b0 0x7f87d413ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.973+0000 7f87d9e00640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87d41604f0 0x7f87d413f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.973+0000 7f87d9400640 1 -- 172.21.15.40:0/1372674973 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87d41604f0 msgr2=0x7f87d413f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.973+0000 7f87d9400640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87d41604f0 0x7f87d413f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.973+0000 7f87d9400640 1 -- 172.21.15.40:0/1372674973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87d41455b0 msgr2=0x7f87d413ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.973+0000 7f87d9400640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87d41455b0 0x7f87d413ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.973+0000 7f87d9400640 1 -- 172.21.15.40:0/1372674973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d413a4d0 con 0x7f87d4071950 2024-03-08T06:04:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.974+0000 7f87d9400640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87d4071950 0x7f87d41f1d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c460 tx=0x7f87c800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.974+0000 7f87d1600640 1 -- 172.21.15.40:0/1372674973 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8016020 con 0x7f87d4071950 2024-03-08T06:04:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.974+0000 7f87d1600640 1 -- 172.21.15.40:0/1372674973 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c8012870 con 0x7f87d4071950 2024-03-08T06:04:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.974+0000 7f87d1600640 1 -- 172.21.15.40:0/1372674973 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8012c60 con 0x7f87d4071950 2024-03-08T06:04:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.974+0000 7f87dbe00640 1 -- 172.21.15.40:0/1372674973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d413a7b0 con 0x7f87d4071950 2024-03-08T06:04:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.974+0000 7f87dbe00640 1 -- 172.21.15.40:0/1372674973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d413ad70 con 0x7f87d4071950 2024-03-08T06:04:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.975+0000 7f87dbe00640 1 -- 172.21.15.40:0/1372674973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d4067cb0 con 0x7f87d4071950 2024-03-08T06:04:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.976+0000 7f87d1600640 1 -- 172.21.15.40:0/1372674973 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87c8019020 con 0x7f87d4071950 2024-03-08T06:04:12.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.977+0000 7f87d1600640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f87b406b4d0 0x7f87b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:12.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.977+0000 7f87d1600640 1 -- 172.21.15.40:0/1372674973 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87c8089ed0 con 0x7f87d4071950 2024-03-08T06:04:12.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.977+0000 7f87d8a00640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f87b406b4d0 0x7f87b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:12.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.978+0000 7f87d8a00640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f87b406b4d0 0x7f87b406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87c0006fd0 tx=0x7f87c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:12.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:12.982+0000 7f87d1600640 1 -- 172.21.15.40:0/1372674973 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87c8020030 con 0x7f87d4071950 2024-03-08T06:04:13.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.189+0000 7f87dbe00640 1 -- 172.21.15.40:0/1372674973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87d40729c0 con 0x7f87d4071950 2024-03-08T06:04:13.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.189+0000 7f87d1600640 1 -- 172.21.15.40:0/1372674973 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87c8058d40 con 0x7f87d4071950 2024-03-08T06:04:13.190 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.192+0000 7f87dbe00640 1 -- 172.21.15.40:0/1372674973 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f87b406b4d0 msgr2=0x7f87b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.192+0000 7f87dbe00640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f87b406b4d0 0x7f87b406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87c0006fd0 tx=0x7f87c0008040 comp rx=0 tx=0).stop 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.192+0000 7f87dbe00640 1 -- 172.21.15.40:0/1372674973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87d4071950 msgr2=0x7f87d41f1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.192+0000 7f87dbe00640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87d4071950 0x7f87d41f1d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c460 tx=0x7f87c800c920 comp rx=0 tx=0).stop 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.193+0000 7f87dbe00640 1 -- 172.21.15.40:0/1372674973 shutdown_connections 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.193+0000 7f87dbe00640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f87b406b4d0 0x7f87b406d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.193+0000 7f87dbe00640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87d41604f0 0x7f87d413f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.193+0000 7f87dbe00640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87d41455b0 0x7f87d413ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.193+0000 7f87dbe00640 1 --2- 172.21.15.40:0/1372674973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87d4071950 0x7f87d41f1d10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.193+0000 7f87dbe00640 1 -- 172.21.15.40:0/1372674973 >> 172.21.15.40:0/1372674973 conn(0x7f87d406ae30 msgr2=0x7f87d40af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.193+0000 7f87dbe00640 1 -- 172.21.15.40:0/1372674973 shutdown_connections 2024-03-08T06:04:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.193+0000 7f87dbe00640 1 -- 172.21.15.40:0/1372674973 wait complete. 2024-03-08T06:04:13.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:13.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.438+0000 7fd602400640 1 Processor -- start 2024-03-08T06:04:13.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.438+0000 7fd602400640 1 -- start start 2024-03-08T06:04:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.440+0000 7fd602400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd5fc1584a0 0x7fd5fc071950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.440+0000 7fd602400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5fc071e90 0x7fd5fc15e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.440+0000 7fd602400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5fc15eb50 0x7fd5fc161a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.440+0000 7fd602400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc15f390 con 0x7fd5fc15eb50 2024-03-08T06:04:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.440+0000 7fd602400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc06d820 con 0x7fd5fc1584a0 2024-03-08T06:04:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.440+0000 7fd602400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc06db00 con 0x7fd5fc071e90 2024-03-08T06:04:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.440+0000 7fd5fbe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5fc15eb50 0x7fd5fc161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.440+0000 7fd5fbe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5fc15eb50 0x7fd5fc161a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45860/0 (socket says 172.21.15.40:45860) 2024-03-08T06:04:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.440+0000 7fd5fbe00640 1 -- 172.21.15.40:0/4218214192 learned_addr learned my addr 172.21.15.40:0/4218214192 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.441+0000 7fd5f2a00640 1 --2- 172.21.15.40:0/4218214192 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5fc071e90 0x7fd5fc15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.441+0000 7fd5fb400640 1 --2- 172.21.15.40:0/4218214192 >> v2:172.21.15.53:3300/0 conn(0x7fd5fc1584a0 0x7fd5fc071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.441+0000 7fd5fbe00640 1 -- 172.21.15.40:0/4218214192 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5fc071e90 msgr2=0x7fd5fc15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.441+0000 7fd5fbe00640 1 --2- 172.21.15.40:0/4218214192 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5fc071e90 0x7fd5fc15e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.441+0000 7fd5fbe00640 1 -- 172.21.15.40:0/4218214192 >> v2:172.21.15.53:3300/0 conn(0x7fd5fc1584a0 msgr2=0x7fd5fc071950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.441+0000 7fd5fbe00640 1 --2- 172.21.15.40:0/4218214192 >> v2:172.21.15.53:3300/0 conn(0x7fd5fc1584a0 0x7fd5fc071950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.441+0000 7fd5fbe00640 1 -- 172.21.15.40:0/4218214192 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc06dde0 con 0x7fd5fc15eb50 2024-03-08T06:04:13.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.442+0000 7fd5fbe00640 1 --2- 172.21.15.40:0/4218214192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5fc15eb50 0x7fd5fc161a90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc072540 tx=0x7fd5f402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998918930936bc86 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:13.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.442+0000 7fd5faa00640 1 -- 172.21.15.40:0/4218214192 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f402e070 con 0x7fd5fc15eb50 2024-03-08T06:04:13.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.442+0000 7fd5faa00640 1 -- 172.21.15.40:0/4218214192 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f4037070 con 0x7fd5fc15eb50 2024-03-08T06:04:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.442+0000 7fd5faa00640 1 -- 172.21.15.40:0/4218214192 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4031a00 con 0x7fd5fc15eb50 2024-03-08T06:04:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.443+0000 7fd602400640 1 -- 172.21.15.40:0/4218214192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5fc15eb50 msgr2=0x7fd5fc161a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.443+0000 7fd602400640 1 --2- 172.21.15.40:0/4218214192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5fc15eb50 0x7fd5fc161a90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc072540 tx=0x7fd5f402b880 comp rx=0 tx=0).stop 2024-03-08T06:04:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.443+0000 7fd602400640 1 -- 172.21.15.40:0/4218214192 shutdown_connections 2024-03-08T06:04:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.443+0000 7fd602400640 1 --2- 172.21.15.40:0/4218214192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5fc15eb50 0x7fd5fc161a90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.443+0000 7fd602400640 1 --2- 172.21.15.40:0/4218214192 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5fc071e90 0x7fd5fc15e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.443+0000 7fd602400640 1 --2- 172.21.15.40:0/4218214192 >> v2:172.21.15.53:3300/0 conn(0x7fd5fc1584a0 0x7fd5fc071950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.443+0000 7fd602400640 1 -- 172.21.15.40:0/4218214192 >> 172.21.15.40:0/4218214192 conn(0x7fd5fc06ae30 msgr2=0x7fd5fc06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.443+0000 7fd602400640 1 -- 172.21.15.40:0/4218214192 shutdown_connections 2024-03-08T06:04:13.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.443+0000 7fd602400640 1 -- 172.21.15.40:0/4218214192 wait complete. 2024-03-08T06:04:13.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.445+0000 7fd602400640 1 Processor -- start 2024-03-08T06:04:13.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.445+0000 7fd602400640 1 -- start start 2024-03-08T06:04:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.446+0000 7fd602400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5fc071e90 0x7fd5fc0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.446+0000 7fd602400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5fc1584a0 0x7fd5fc13b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.446+0000 7fd602400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5fc15eb50 0x7fd5fc15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.446+0000 7fd602400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc06e5e0 con 0x7fd5fc1584a0 2024-03-08T06:04:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.446+0000 7fd602400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc15ef70 con 0x7fd5fc071e90 2024-03-08T06:04:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.446+0000 7fd602400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0b0890 con 0x7fd5fc15eb50 2024-03-08T06:04:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.446+0000 7fd5f2a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5fc1584a0 0x7fd5fc13b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.447+0000 7fd5f2a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5fc1584a0 0x7fd5fc13b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45874/0 (socket says 172.21.15.40:45874) 2024-03-08T06:04:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.447+0000 7fd5f2a00640 1 -- 172.21.15.40:0/1312972496 learned_addr learned my addr 172.21.15.40:0/1312972496 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.447+0000 7fd5fb400640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5fc071e90 0x7fd5fc0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.447+0000 7fd5fbe00640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5fc15eb50 0x7fd5fc15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.447+0000 7fd5f2a00640 1 -- 172.21.15.40:0/1312972496 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5fc15eb50 msgr2=0x7fd5fc15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:13.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.447+0000 7fd5f2a00640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5fc15eb50 0x7fd5fc15b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.447+0000 7fd5f2a00640 1 -- 172.21.15.40:0/1312972496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5fc071e90 msgr2=0x7fd5fc0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:13.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.447+0000 7fd5f2a00640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5fc071e90 0x7fd5fc0b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.447+0000 7fd5f2a00640 1 -- 172.21.15.40:0/1312972496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc0b02d0 con 0x7fd5fc1584a0 2024-03-08T06:04:13.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.447+0000 7fd5f2a00640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5fc1584a0 0x7fd5fc13b380 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400d800 tx=0x7fd5e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:13.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.447+0000 7fd5f3e00640 1 -- 172.21.15.40:0/1312972496 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4018030 con 0x7fd5fc1584a0 2024-03-08T06:04:13.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.448+0000 7fd5f3e00640 1 -- 172.21.15.40:0/1312972496 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e40096a0 con 0x7fd5fc1584a0 2024-03-08T06:04:13.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.448+0000 7fd5f3e00640 1 -- 172.21.15.40:0/1312972496 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4009870 con 0x7fd5fc1584a0 2024-03-08T06:04:13.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.448+0000 7fd602400640 1 -- 172.21.15.40:0/1312972496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc0b05b0 con 0x7fd5fc1584a0 2024-03-08T06:04:13.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.448+0000 7fd602400640 1 -- 172.21.15.40:0/1312972496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc0b38b0 con 0x7fd5fc1584a0 2024-03-08T06:04:13.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.449+0000 7fd602400640 1 -- 172.21.15.40:0/1312972496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc067cb0 con 0x7fd5fc1584a0 2024-03-08T06:04:13.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.450+0000 7fd5f3e00640 1 -- 172.21.15.40:0/1312972496 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5e401b070 con 0x7fd5fc1584a0 2024-03-08T06:04:13.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.451+0000 7fd5f3e00640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d806b4d0 0x7fd5d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.451+0000 7fd5f3e00640 1 -- 172.21.15.40:0/1312972496 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5e408a4c0 con 0x7fd5fc1584a0 2024-03-08T06:04:13.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.451+0000 7fd5fb400640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d806b4d0 0x7fd5d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.452+0000 7fd5fb400640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d806b4d0 0x7fd5d806d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0099c0 tx=0x7fd5ec003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:13.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.455+0000 7fd5f3e00640 1 -- 172.21.15.40:0/1312972496 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5e4059330 con 0x7fd5fc1584a0 2024-03-08T06:04:13.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.662+0000 7fd602400640 1 -- 172.21.15.40:0/1312972496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5fc06f280 con 0x7fd5fc1584a0 2024-03-08T06:04:13.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.663+0000 7fd5f3e00640 1 -- 172.21.15.40:0/1312972496 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5e4058c30 con 0x7fd5fc1584a0 2024-03-08T06:04:13.664 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:13.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 -- 172.21.15.40:0/1312972496 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d806b4d0 msgr2=0x7fd5d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:13.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d806b4d0 0x7fd5d806d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0099c0 tx=0x7fd5ec003990 comp rx=0 tx=0).stop 2024-03-08T06:04:13.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 -- 172.21.15.40:0/1312972496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5fc1584a0 msgr2=0x7fd5fc13b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:13.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5fc1584a0 0x7fd5fc13b380 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400d800 tx=0x7fd5e400dcc0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 -- 172.21.15.40:0/1312972496 shutdown_connections 2024-03-08T06:04:13.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d806b4d0 0x7fd5d806d990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5fc15eb50 0x7fd5fc15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5fc1584a0 0x7fd5fc13b380 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 --2- 172.21.15.40:0/1312972496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5fc071e90 0x7fd5fc0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 -- 172.21.15.40:0/1312972496 >> 172.21.15.40:0/1312972496 conn(0x7fd5fc06ae30 msgr2=0x7fd5fc1611a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:13.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 -- 172.21.15.40:0/1312972496 shutdown_connections 2024-03-08T06:04:13.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.666+0000 7fd602400640 1 -- 172.21.15.40:0/1312972496 wait complete. 2024-03-08T06:04:13.685 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:04:13.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.908+0000 7f21a7800640 1 Processor -- start 2024-03-08T06:04:13.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.908+0000 7f21a7800640 1 -- start start 2024-03-08T06:04:13.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.910+0000 7f21a7800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21a0144030 0x7f21a00cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.910+0000 7f21a7800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21a015e580 0x7f21a006e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.910+0000 7f21a7800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a015eaf0 con 0x7f21a006ec50 2024-03-08T06:04:13.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.910+0000 7f21a7800640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a006f0b0 con 0x7f21a0144030 2024-03-08T06:04:13.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.910+0000 7f21a7800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00b3350 con 0x7f21a015e580 2024-03-08T06:04:13.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.911+0000 7f21a4e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21a0144030 0x7f21a00cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.911+0000 7f21a5800640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f21a006ec50 0x7f21a0071840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60974/0 (socket says 172.21.15.40:60974) 2024-03-08T06:04:13.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.911+0000 7f21a4e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21a0144030 0x7f21a00cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45888/0 (socket says 172.21.15.40:45888) 2024-03-08T06:04:13.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.911+0000 7f21a5800640 1 -- 172.21.15.40:0/2068013737 learned_addr learned my addr 172.21.15.40:0/2068013737 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:13.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.911+0000 7f219fe00640 1 --2- 172.21.15.40:0/2068013737 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21a015e580 0x7f21a006e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.911+0000 7f21a4e00640 1 -- 172.21.15.40:0/2068013737 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21a015e580 msgr2=0x7f21a006e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:13.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.911+0000 7f21a4e00640 1 --2- 172.21.15.40:0/2068013737 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21a015e580 0x7f21a006e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.911+0000 7f21a4e00640 1 -- 172.21.15.40:0/2068013737 >> v1:172.21.15.40:6789/0 conn(0x7f21a006ec50 legacy=0x7f21a0071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:13.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.911+0000 7f21a4e00640 1 -- 172.21.15.40:0/2068013737 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a015edd0 con 0x7f21a0144030 2024-03-08T06:04:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.912+0000 7f21a4e00640 1 --2- 172.21.15.40:0/2068013737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a0144030 0x7f21a00cf080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f21940097a0 tx=0x7f219402dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e49b476f8c29ee server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.913+0000 7f219f400640 1 -- 172.21.15.40:0/2068013737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219402f880 con 0x7f21a0144030 2024-03-08T06:04:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.913+0000 7f219f400640 1 -- 172.21.15.40:0/2068013737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219402fa20 con 0x7f21a0144030 2024-03-08T06:04:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.913+0000 7f219f400640 1 -- 172.21.15.40:0/2068013737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219402fdf0 con 0x7f21a0144030 2024-03-08T06:04:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.913+0000 7f21a7800640 1 -- 172.21.15.40:0/2068013737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a0144030 msgr2=0x7f21a00cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.913+0000 7f21a7800640 1 --2- 172.21.15.40:0/2068013737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a0144030 0x7f21a00cf080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f21940097a0 tx=0x7f219402dac0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.913+0000 7f21a7800640 1 -- 172.21.15.40:0/2068013737 shutdown_connections 2024-03-08T06:04:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.913+0000 7f21a7800640 1 --2- 172.21.15.40:0/2068013737 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21a015e580 0x7f21a006e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.913+0000 7f21a7800640 1 --2- 172.21.15.40:0/2068013737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a0144030 0x7f21a00cf080 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.913+0000 7f21a7800640 1 -- 172.21.15.40:0/2068013737 >> 172.21.15.40:0/2068013737 conn(0x7f21a006ae30 msgr2=0x7f21a006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.913+0000 7f21a7800640 1 -- 172.21.15.40:0/2068013737 shutdown_connections 2024-03-08T06:04:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.913+0000 7f21a7800640 1 -- 172.21.15.40:0/2068013737 wait complete. 2024-03-08T06:04:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.915+0000 7f21a7800640 1 Processor -- start 2024-03-08T06:04:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.915+0000 7f21a7800640 1 -- start start 2024-03-08T06:04:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.916+0000 7f21a7800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21a006ec50 0x7f21a0141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.916+0000 7f21a7800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0144030 0x7f21a013e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.916+0000 7f21a7800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a015e580 0x7f21a013fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.916+0000 7f21a7800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00cf5c0 con 0x7f21a015e580 2024-03-08T06:04:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.916+0000 7f21a7800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a015f390 con 0x7f21a0144030 2024-03-08T06:04:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.916+0000 7f21a7800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00cf740 con 0x7f21a006ec50 2024-03-08T06:04:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.917+0000 7f21a5800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a015e580 0x7f21a013fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.917+0000 7f21a5800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a015e580 0x7f21a013fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45890/0 (socket says 172.21.15.40:45890) 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.917+0000 7f21a5800640 1 -- 172.21.15.40:0/1632951836 learned_addr learned my addr 172.21.15.40:0/1632951836 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.917+0000 7f21a4e00640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21a006ec50 0x7f21a0141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.917+0000 7f219fe00640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0144030 0x7f21a013e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.917+0000 7f21a5800640 1 -- 172.21.15.40:0/1632951836 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21a006ec50 msgr2=0x7f21a0141520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.917+0000 7f21a5800640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21a006ec50 0x7f21a0141520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.917+0000 7f21a5800640 1 -- 172.21.15.40:0/1632951836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0144030 msgr2=0x7f21a013e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.917+0000 7f21a5800640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0144030 0x7f21a013e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.917+0000 7f21a5800640 1 -- 172.21.15.40:0/1632951836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a0140480 con 0x7f21a015e580 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.917+0000 7f21a5800640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a015e580 0x7f21a013fac0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f219800caa0 tx=0x7f219800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.918+0000 7f219cc00640 1 -- 172.21.15.40:0/1632951836 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198017070 con 0x7f21a015e580 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.918+0000 7f219cc00640 1 -- 172.21.15.40:0/1632951836 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219800ac00 con 0x7f21a015e580 2024-03-08T06:04:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.918+0000 7f21a7800640 1 -- 172.21.15.40:0/1632951836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a0140760 con 0x7f21a015e580 2024-03-08T06:04:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.918+0000 7f219cc00640 1 -- 172.21.15.40:0/1632951836 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198012620 con 0x7f21a015e580 2024-03-08T06:04:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.918+0000 7f21a7800640 1 -- 172.21.15.40:0/1632951836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a0141000 con 0x7f21a015e580 2024-03-08T06:04:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.919+0000 7f21a7800640 1 -- 172.21.15.40:0/1632951836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a0067cb0 con 0x7f21a015e580 2024-03-08T06:04:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.920+0000 7f219cc00640 1 -- 172.21.15.40:0/1632951836 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f219801a070 con 0x7f21a015e580 2024-03-08T06:04:13.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.921+0000 7f219cc00640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f218006b520 0x7f218006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:13.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.921+0000 7f219cc00640 1 -- 172.21.15.40:0/1632951836 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2198089f10 con 0x7f21a015e580 2024-03-08T06:04:13.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.921+0000 7f21a4e00640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f218006b520 0x7f218006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:13.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.922+0000 7f21a4e00640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f218006b520 0x7f218006d9e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2194038000 tx=0x7f219403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:13.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:13.926+0000 7f219cc00640 1 -- 172.21.15.40:0/1632951836 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2198058e00 con 0x7f21a015e580 2024-03-08T06:04:14.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.161+0000 7f21a7800640 1 -- 172.21.15.40:0/1632951836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21a00aeec0 con 0x7f21a015e580 2024-03-08T06:04:14.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.162+0000 7f219cc00640 1 -- 172.21.15.40:0/1632951836 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f2198058700 con 0x7f21a015e580 2024-03-08T06:04:14.162 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:14.163 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3266388900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3266388900},{"type":"v1","addr":"172.21.15.40:6837","nonce":3266388900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2471668569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2471668569},{"type":"v1","addr":"172.21.15.53:6834","nonce":2471668569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3262228631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3262228631},{"type":"v1","addr":"172.21.15.53:6835","nonce":3262228631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:04:02.003553+0000","modified":"2024-03-08T06:04:05.045865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3709744112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3709744112},{"type":"v1","addr":"172.21.15.40:6835","nonce":3709744112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:04:14.163 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-03-08T06:04:14.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 -- 172.21.15.40:0/1632951836 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f218006b520 msgr2=0x7f218006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:14.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f218006b520 0x7f218006d9e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2194038000 tx=0x7f219403b040 comp rx=0 tx=0).stop 2024-03-08T06:04:14.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 -- 172.21.15.40:0/1632951836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a015e580 msgr2=0x7f21a013fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:14.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a015e580 0x7f21a013fac0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f219800caa0 tx=0x7f219800cf60 comp rx=0 tx=0).stop 2024-03-08T06:04:14.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 -- 172.21.15.40:0/1632951836 shutdown_connections 2024-03-08T06:04:14.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f218006b520 0x7f218006d9e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a015e580 0x7f21a013fac0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0144030 0x7f21a013e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 --2- 172.21.15.40:0/1632951836 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21a006ec50 0x7f21a0141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 -- 172.21.15.40:0/1632951836 >> 172.21.15.40:0/1632951836 conn(0x7f21a006ae30 msgr2=0x7f21a00b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:14.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 -- 172.21.15.40:0/1632951836 shutdown_connections 2024-03-08T06:04:14.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.165+0000 7f21a7800640 1 -- 172.21.15.40:0/1632951836 wait complete. 2024-03-08T06:04:14.184 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.406+0000 7f789de00640 1 Processor -- start 2024-03-08T06:04:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.407+0000 7f789de00640 1 -- start start 2024-03-08T06:04:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.408+0000 7f789de00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f789815de70 0x7f78980cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.408+0000 7f789de00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f789806cc20 0x7f78980b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.408+0000 7f789de00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78980cf0c0 con 0x7f78980b5bb0 2024-03-08T06:04:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.408+0000 7f789de00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78980b6020 con 0x7f789815de70 2024-03-08T06:04:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.408+0000 7f789de00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78980b93e0 con 0x7f789806cc20 2024-03-08T06:04:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.409+0000 7f7896a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f789815de70 0x7f78980cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.409+0000 7f7896a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f789815de70 0x7f78980cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45906/0 (socket says 172.21.15.40:45906) 2024-03-08T06:04:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.409+0000 7f7896a00640 1 -- 172.21.15.40:0/2043392629 learned_addr learned my addr 172.21.15.40:0/2043392629 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.409+0000 7f7896000640 1 --2- 172.21.15.40:0/2043392629 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f789806cc20 0x7f78980b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.409+0000 7f7896a00640 1 -- 172.21.15.40:0/2043392629 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f789806cc20 msgr2=0x7f78980b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.409+0000 7f7896a00640 1 --2- 172.21.15.40:0/2043392629 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f789806cc20 0x7f78980b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.409+0000 7f7896a00640 1 -- 172.21.15.40:0/2043392629 >> v1:172.21.15.53:6789/0 conn(0x7f78980b5bb0 legacy=0x7f78981619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.409+0000 7f7896a00640 1 -- 172.21.15.40:0/2043392629 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78980b96c0 con 0x7f789815de70 2024-03-08T06:04:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.409+0000 7f7896000640 1 --2- 172.21.15.40:0/2043392629 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f789806cc20 0x7f78980b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.410+0000 7f7896a00640 1 --2- 172.21.15.40:0/2043392629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789815de70 0x7f78980cea30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7880009870 tx=0x7f788002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6578be119e2b95e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.411+0000 7f7895600640 1 -- 172.21.15.40:0/2043392629 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880034070 con 0x7f789815de70 2024-03-08T06:04:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.411+0000 7f7895600640 1 -- 172.21.15.40:0/2043392629 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7880030840 con 0x7f789815de70 2024-03-08T06:04:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.411+0000 7f7895600640 1 -- 172.21.15.40:0/2043392629 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880030c10 con 0x7f789815de70 2024-03-08T06:04:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.411+0000 7f789de00640 1 -- 172.21.15.40:0/2043392629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789815de70 msgr2=0x7f78980cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.411+0000 7f789de00640 1 --2- 172.21.15.40:0/2043392629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789815de70 0x7f78980cea30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7880009870 tx=0x7f788002b980 comp rx=0 tx=0).stop 2024-03-08T06:04:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.411+0000 7f789de00640 1 -- 172.21.15.40:0/2043392629 shutdown_connections 2024-03-08T06:04:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.411+0000 7f789de00640 1 --2- 172.21.15.40:0/2043392629 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f789806cc20 0x7f78980b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.411+0000 7f789de00640 1 --2- 172.21.15.40:0/2043392629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789815de70 0x7f78980cea30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.411+0000 7f789de00640 1 -- 172.21.15.40:0/2043392629 >> 172.21.15.40:0/2043392629 conn(0x7f78980d0850 msgr2=0x7f78980d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.412+0000 7f789de00640 1 -- 172.21.15.40:0/2043392629 shutdown_connections 2024-03-08T06:04:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.412+0000 7f789de00640 1 -- 172.21.15.40:0/2043392629 wait complete. 2024-03-08T06:04:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.414+0000 7f789de00640 1 Processor -- start 2024-03-08T06:04:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.414+0000 7f789de00640 1 -- start start 2024-03-08T06:04:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.415+0000 7f789de00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f789806cc20 0x7f78980d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.415+0000 7f789de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78980b5bb0 0x7f789815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.415+0000 7f789de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789815de70 0x7f789815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.415+0000 7f789de00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789806d060 con 0x7f789815de70 2024-03-08T06:04:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.415+0000 7f789de00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789815e250 con 0x7f78980b5bb0 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.415+0000 7f789de00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78980d3060 con 0x7f789806cc20 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.415+0000 7f7897400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789815de70 0x7f789815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.416+0000 7f7897400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789815de70 0x7f789815c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45922/0 (socket says 172.21.15.40:45922) 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.416+0000 7f7897400640 1 -- 172.21.15.40:0/1952515883 learned_addr learned my addr 172.21.15.40:0/1952515883 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.416+0000 7f7896000640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78980b5bb0 0x7f789815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.416+0000 7f7896a00640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f789806cc20 0x7f78980d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.416+0000 7f7897400640 1 -- 172.21.15.40:0/1952515883 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f789806cc20 msgr2=0x7f78980d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.416+0000 7f7897400640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f789806cc20 0x7f78980d45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.416+0000 7f7897400640 1 -- 172.21.15.40:0/1952515883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78980b5bb0 msgr2=0x7f789815ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.416+0000 7f7897400640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78980b5bb0 0x7f789815ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.416+0000 7f7897400640 1 -- 172.21.15.40:0/1952515883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78980caf10 con 0x7f789815de70 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.416+0000 7f7896000640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78980b5bb0 0x7f789815ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.416+0000 7f7897400640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789815de70 0x7f789815c210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f789000c780 tx=0x7f789000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:14.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.417+0000 7f788f400640 1 -- 172.21.15.40:0/1952515883 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890004b60 con 0x7f789815de70 2024-03-08T06:04:14.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.417+0000 7f788f400640 1 -- 172.21.15.40:0/1952515883 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7890004d00 con 0x7f789815de70 2024-03-08T06:04:14.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.417+0000 7f788f400640 1 -- 172.21.15.40:0/1952515883 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789000aa90 con 0x7f789815de70 2024-03-08T06:04:14.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.417+0000 7f789de00640 1 -- 172.21.15.40:0/1952515883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78980cc8e0 con 0x7f789815de70 2024-03-08T06:04:14.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.417+0000 7f789de00640 1 -- 172.21.15.40:0/1952515883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78980ccea0 con 0x7f789815de70 2024-03-08T06:04:14.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.418+0000 7f789de00640 1 -- 172.21.15.40:0/1952515883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78980cd0a0 con 0x7f789815de70 2024-03-08T06:04:14.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.419+0000 7f788f400640 1 -- 172.21.15.40:0/1952515883 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f789000ac30 con 0x7f789815de70 2024-03-08T06:04:14.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.420+0000 7f788f400640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f787406b4d0 0x7f787406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.420+0000 7f788f400640 1 -- 172.21.15.40:0/1952515883 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f789008a440 con 0x7f789815de70 2024-03-08T06:04:14.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.420+0000 7f7896a00640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f787406b4d0 0x7f787406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.421+0000 7f7896a00640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f787406b4d0 0x7f787406d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7880036000 tx=0x7f788003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.425+0000 7f788f400640 1 -- 172.21.15.40:0/1952515883 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78980cd0a0 con 0x7f789815de70 2024-03-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.632+0000 7f789de00640 1 -- 172.21.15.40:0/1952515883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78980d1ad0 con 0x7f789815de70 2024-03-08T06:04:14.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.635+0000 7f788f400640 1 -- 172.21.15.40:0/1952515883 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f7890020070 con 0x7f789815de70 2024-03-08T06:04:14.635 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:14.635 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:14.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 -- 172.21.15.40:0/1952515883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f787406b4d0 msgr2=0x7f787406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:14.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f787406b4d0 0x7f787406d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7880036000 tx=0x7f788003b040 comp rx=0 tx=0).stop 2024-03-08T06:04:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 -- 172.21.15.40:0/1952515883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789815de70 msgr2=0x7f789815c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789815de70 0x7f789815c210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f789000c780 tx=0x7f789000cc40 comp rx=0 tx=0).stop 2024-03-08T06:04:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 -- 172.21.15.40:0/1952515883 shutdown_connections 2024-03-08T06:04:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f787406b4d0 0x7f787406d990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789815de70 0x7f789815c210 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78980b5bb0 0x7f789815ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 --2- 172.21.15.40:0/1952515883 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f789806cc20 0x7f78980d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 -- 172.21.15.40:0/1952515883 >> 172.21.15.40:0/1952515883 conn(0x7f78980d0850 msgr2=0x7f78980d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 -- 172.21.15.40:0/1952515883 shutdown_connections 2024-03-08T06:04:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.638+0000 7f789de00640 1 -- 172.21.15.40:0/1952515883 wait complete. 2024-03-08T06:04:14.656 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:14.656 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:14.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.879+0000 7f371fc00640 1 Processor -- start 2024-03-08T06:04:14.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.879+0000 7f371fc00640 1 -- start start 2024-03-08T06:04:14.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.880+0000 7f371fc00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f371815f7a0 0x7f3718099db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.881+0000 7f371fc00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37180cce50 0x7f37180b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.881+0000 7f371fc00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371809a470 con 0x7f371815f3c0 2024-03-08T06:04:14.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.881+0000 7f371fc00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37180b3360 con 0x7f37180cce50 2024-03-08T06:04:14.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.881+0000 7f371fc00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371809a750 con 0x7f371815f7a0 2024-03-08T06:04:14.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.881+0000 7f371dc00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37180cce50 0x7f37180b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.881+0000 7f371dc00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37180cce50 0x7f37180b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45930/0 (socket says 172.21.15.40:45930) 2024-03-08T06:04:14.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.881+0000 7f371dc00640 1 -- 172.21.15.40:0/1975006478 learned_addr learned my addr 172.21.15.40:0/1975006478 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:14.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.881+0000 7f3717e00640 1 --2- 172.21.15.40:0/1975006478 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f371815f7a0 0x7f3718099db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.881+0000 7f371dc00640 1 -- 172.21.15.40:0/1975006478 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f371815f7a0 msgr2=0x7f3718099db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:14.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.881+0000 7f371dc00640 1 --2- 172.21.15.40:0/1975006478 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f371815f7a0 0x7f3718099db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.881+0000 7f371dc00640 1 -- 172.21.15.40:0/1975006478 >> v1:172.21.15.53:6789/0 conn(0x7f371815f3c0 legacy=0x7f37180cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:14.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.882+0000 7f371dc00640 1 -- 172.21.15.40:0/1975006478 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371809aa30 con 0x7f37180cce50 2024-03-08T06:04:14.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.883+0000 7f371dc00640 1 --2- 172.21.15.40:0/1975006478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37180cce50 0x7f37180b2e20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3710009e50 tx=0x7f371002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6286f19408a23e17 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:14.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.883+0000 7f3717400640 1 -- 172.21.15.40:0/1975006478 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371002ca70 con 0x7f37180cce50 2024-03-08T06:04:14.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.883+0000 7f3717400640 1 -- 172.21.15.40:0/1975006478 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371002cc10 con 0x7f37180cce50 2024-03-08T06:04:14.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.883+0000 7f3717400640 1 -- 172.21.15.40:0/1975006478 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37100343f0 con 0x7f37180cce50 2024-03-08T06:04:14.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.884+0000 7f371fc00640 1 -- 172.21.15.40:0/1975006478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37180cce50 msgr2=0x7f37180b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:14.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.884+0000 7f371fc00640 1 --2- 172.21.15.40:0/1975006478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37180cce50 0x7f37180b2e20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3710009e50 tx=0x7f371002bed0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.884+0000 7f371fc00640 1 -- 172.21.15.40:0/1975006478 shutdown_connections 2024-03-08T06:04:14.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.884+0000 7f371fc00640 1 --2- 172.21.15.40:0/1975006478 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f371815f7a0 0x7f3718099db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.884+0000 7f371fc00640 1 --2- 172.21.15.40:0/1975006478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37180cce50 0x7f37180b2e20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.884+0000 7f371fc00640 1 -- 172.21.15.40:0/1975006478 >> 172.21.15.40:0/1975006478 conn(0x7f37180b3620 msgr2=0x7f37180b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:14.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.884+0000 7f371fc00640 1 -- 172.21.15.40:0/1975006478 shutdown_connections 2024-03-08T06:04:14.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.884+0000 7f371fc00640 1 -- 172.21.15.40:0/1975006478 wait complete. 2024-03-08T06:04:14.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.886+0000 7f371fc00640 1 Processor -- start 2024-03-08T06:04:14.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.886+0000 7f371fc00640 1 -- start start 2024-03-08T06:04:14.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.887+0000 7f371fc00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37180cce50 0x7f37181f1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.887+0000 7f371fc00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f371815f3c0 0x7f37181f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.887+0000 7f371fc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371815f7a0 0x7f37180b6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.887+0000 7f371fc00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37180c5780 con 0x7f371815f3c0 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.887+0000 7f371fc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371809aff0 con 0x7f371815f7a0 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.887+0000 7f371fc00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37180c5900 con 0x7f37180cce50 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.887+0000 7f3717e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f371815f3c0 0x7f37181f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.887+0000 7f3717e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f371815f3c0 0x7f37181f28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45932/0 (socket says 172.21.15.40:45932) 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.887+0000 7f3717e00640 1 -- 172.21.15.40:0/3988372242 learned_addr learned my addr 172.21.15.40:0/3988372242 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.887+0000 7f371d200640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37180cce50 0x7f37181f1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.887+0000 7f371dc00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371815f7a0 0x7f37180b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f3717e00640 1 -- 172.21.15.40:0/3988372242 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37180cce50 msgr2=0x7f37181f1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f3717e00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37180cce50 0x7f37181f1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f3717e00640 1 -- 172.21.15.40:0/3988372242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371815f7a0 msgr2=0x7f37180b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f3717e00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371815f7a0 0x7f37180b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f3717e00640 1 -- 172.21.15.40:0/3988372242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37180b7150 con 0x7f371815f3c0 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f371dc00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371815f7a0 0x7f37180b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f3717e00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f371815f3c0 0x7f37181f28b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f370400d5f0 tx=0x7f370400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:14.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f3714c00640 1 -- 172.21.15.40:0/3988372242 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040125f0 con 0x7f371815f3c0 2024-03-08T06:04:14.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f3714c00640 1 -- 172.21.15.40:0/3988372242 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3704019070 con 0x7f371815f3c0 2024-03-08T06:04:14.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f3714c00640 1 -- 172.21.15.40:0/3988372242 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040129d0 con 0x7f371815f3c0 2024-03-08T06:04:14.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f371fc00640 1 -- 172.21.15.40:0/3988372242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37180b7430 con 0x7f371815f3c0 2024-03-08T06:04:14.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.888+0000 7f371fc00640 1 -- 172.21.15.40:0/3988372242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37180b4980 con 0x7f371815f3c0 2024-03-08T06:04:14.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.890+0000 7f371fc00640 1 -- 172.21.15.40:0/3988372242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37180683c0 con 0x7f371815f3c0 2024-03-08T06:04:14.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.891+0000 7f3714c00640 1 -- 172.21.15.40:0/3988372242 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3704012b70 con 0x7f371815f3c0 2024-03-08T06:04:14.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.891+0000 7f3714c00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f36f806b4d0 0x7f36f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:14.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.892+0000 7f3714c00640 1 -- 172.21.15.40:0/3988372242 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f370408a5f0 con 0x7f371815f3c0 2024-03-08T06:04:14.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.892+0000 7f371d200640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f36f806b4d0 0x7f36f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:14.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.892+0000 7f371d200640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f36f806b4d0 0x7f36f806d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f371809fb50 tx=0x7f370c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:14.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:14.896+0000 7f3714c00640 1 -- 172.21.15.40:0/3988372242 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3704059470 con 0x7f371815f3c0 2024-03-08T06:04:15.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.100+0000 7f371fc00640 1 -- 172.21.15.40:0/3988372242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3718099f30 con 0x7f371815f3c0 2024-03-08T06:04:15.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.101+0000 7f3714c00640 1 -- 172.21.15.40:0/3988372242 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3704058d70 con 0x7f371815f3c0 2024-03-08T06:04:15.101 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:15.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.104+0000 7f371fc00640 1 -- 172.21.15.40:0/3988372242 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f36f806b4d0 msgr2=0x7f36f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.104+0000 7f371fc00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f36f806b4d0 0x7f36f806d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f371809fb50 tx=0x7f370c009040 comp rx=0 tx=0).stop 2024-03-08T06:04:15.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.104+0000 7f371fc00640 1 -- 172.21.15.40:0/3988372242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f371815f3c0 msgr2=0x7f37181f28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.104+0000 7f371fc00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f371815f3c0 0x7f37181f28b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f370400d5f0 tx=0x7f370400dab0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.104+0000 7f371fc00640 1 -- 172.21.15.40:0/3988372242 shutdown_connections 2024-03-08T06:04:15.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.104+0000 7f371fc00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f36f806b4d0 0x7f36f806d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.104+0000 7f371fc00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371815f7a0 0x7f37180b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.104+0000 7f371fc00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f371815f3c0 0x7f37181f28b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.104+0000 7f371fc00640 1 --2- 172.21.15.40:0/3988372242 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37180cce50 0x7f37181f1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.104+0000 7f371fc00640 1 -- 172.21.15.40:0/3988372242 >> 172.21.15.40:0/3988372242 conn(0x7f37180b3620 msgr2=0x7f37180d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:15.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.104+0000 7f371fc00640 1 -- 172.21.15.40:0/3988372242 shutdown_connections 2024-03-08T06:04:15.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.105+0000 7f371fc00640 1 -- 172.21.15.40:0/3988372242 wait complete. 2024-03-08T06:04:15.122 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:15.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.334+0000 7fb806e00640 1 Processor -- start 2024-03-08T06:04:15.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.335+0000 7fb806e00640 1 -- start start 2024-03-08T06:04:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.336+0000 7fb806e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb80015de60 0x7fb800161930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.336+0000 7fb806e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80015e240 con 0x7fb8000cf030 2024-03-08T06:04:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.336+0000 7fb806e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb800162030 con 0x7fb8000b9630 2024-03-08T06:04:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.336+0000 7fb806e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb800162310 con 0x7fb80015de60 2024-03-08T06:04:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.336+0000 7fb806400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb8000cf030 0x7fb8000d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60978/0 (socket says 172.21.15.40:60978) 2024-03-08T06:04:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb806400640 1 -- 172.21.15.40:0/1243010050 learned_addr learned my addr 172.21.15.40:0/1243010050 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb805a00640 1 --2- 172.21.15.40:0/1243010050 >> v2:172.21.15.53:3300/0 conn(0x7fb80015de60 0x7fb800161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431803632 0 0) 0x7fb80015e240 con 0x7fb8000cf030 2024-03-08T06:04:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e4003520 con 0x7fb8000cf030 2024-03-08T06:04:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2169563422 0 0) 0x7fb7e4003520 con 0x7fb8000cf030 2024-03-08T06:04:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 >> v2:172.21.15.53:3300/0 conn(0x7fb80015de60 msgr2=0x7fb800161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 --2- 172.21.15.40:0/1243010050 >> v2:172.21.15.53:3300/0 conn(0x7fb80015de60 0x7fb800161930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 >> v1:172.21.15.53:6789/0 conn(0x7fb8000b9630 legacy=0x7fb8000ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8001625f0 con 0x7fb8000cf030 2024-03-08T06:04:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fb7f8001670 con 0x7fb8000cf030 2024-03-08T06:04:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 >> v1:172.21.15.40:6789/0 conn(0x7fb8000cf030 legacy=0x7fb8000d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 --2- 172.21.15.40:0/1243010050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7e4003a20 0x7fb7e4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 --2- 172.21.15.40:0/1243010050 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7e4004800 0x7fb7e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.337+0000 7fb7ffe00640 1 --2- 172.21.15.40:0/1243010050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 0x7fb7e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.338+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb800162310 con 0x7fb7e40051c0 2024-03-08T06:04:15.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.338+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e400c0f0 con 0x7fb7e4003a20 2024-03-08T06:04:15.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.338+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e400c2f0 con 0x7fb7e4004800 2024-03-08T06:04:15.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.338+0000 7fb806400640 1 --2- 172.21.15.40:0/1243010050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 0x7fb7e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.338+0000 7fb805a00640 1 --2- 172.21.15.40:0/1243010050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7e4003a20 0x7fb7e4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.338+0000 7fb805000640 1 --2- 172.21.15.40:0/1243010050 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7e4004800 0x7fb7e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.338+0000 7fb806400640 1 -- 172.21.15.40:0/1243010050 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7e4004800 msgr2=0x7fb7e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:15.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.338+0000 7fb806400640 1 --2- 172.21.15.40:0/1243010050 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7e4004800 0x7fb7e4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.338+0000 7fb806400640 1 -- 172.21.15.40:0/1243010050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7e4003a20 msgr2=0x7fb7e4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:15.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.338+0000 7fb806400640 1 --2- 172.21.15.40:0/1243010050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7e4003a20 0x7fb7e4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.338+0000 7fb806400640 1 -- 172.21.15.40:0/1243010050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e4003520 con 0x7fb7e40051c0 2024-03-08T06:04:15.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.339+0000 7fb806400640 1 -- 172.21.15.40:0/1243010050 reap_dead start 2024-03-08T06:04:15.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.339+0000 7fb806400640 1 --2- 172.21.15.40:0/1243010050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 0x7fb7e400b9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8000b6a70 tx=0x7fb7e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41db72bad525e38f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:15.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.340+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8007d10 con 0x7fb7e40051c0 2024-03-08T06:04:15.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.340+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f8030070 con 0x7fb7e40051c0 2024-03-08T06:04:15.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.340+0000 7fb7ffe00640 1 -- 172.21.15.40:0/1243010050 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8005cf0 con 0x7fb7e40051c0 2024-03-08T06:04:15.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.340+0000 7fb806e00640 1 -- 172.21.15.40:0/1243010050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 msgr2=0x7fb7e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.340+0000 7fb806e00640 1 --2- 172.21.15.40:0/1243010050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 0x7fb7e400b9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8000b6a70 tx=0x7fb7e4005830 comp rx=0 tx=0).stop 2024-03-08T06:04:15.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.340+0000 7fb806e00640 1 -- 172.21.15.40:0/1243010050 shutdown_connections 2024-03-08T06:04:15.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.340+0000 7fb806e00640 1 --2- 172.21.15.40:0/1243010050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 0x7fb7e400b9c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.340+0000 7fb806e00640 1 -- 172.21.15.40:0/1243010050 >> 172.21.15.40:0/1243010050 conn(0x7fb80006c500 msgr2=0x7fb80006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:15.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.340+0000 7fb806e00640 1 -- 172.21.15.40:0/1243010050 shutdown_connections 2024-03-08T06:04:15.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.340+0000 7fb806e00640 1 -- 172.21.15.40:0/1243010050 wait complete. 2024-03-08T06:04:15.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.342+0000 7fb806e00640 1 Processor -- start 2024-03-08T06:04:15.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.342+0000 7fb806e00640 1 -- start start 2024-03-08T06:04:15.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.343+0000 7fb806e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 0x7fb80015c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.343+0000 7fb806e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8000cc060 0x7fb8000ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.343+0000 7fb806e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8000cd150 0x7fb8000d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.343+0000 7fb806e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb800161930 con 0x7fb7e40051c0 2024-03-08T06:04:15.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.343+0000 7fb806e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8000b9a90 con 0x7fb8000cd150 2024-03-08T06:04:15.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.343+0000 7fb806e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb800161ab0 con 0x7fb8000cc060 2024-03-08T06:04:15.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.344+0000 7fb805a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 0x7fb80015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.344+0000 7fb805a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 0x7fb80015c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45952/0 (socket says 172.21.15.40:45952) 2024-03-08T06:04:15.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.344+0000 7fb805a00640 1 -- 172.21.15.40:0/2333004686 learned_addr learned my addr 172.21.15.40:0/2333004686 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:15.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.344+0000 7fb806400640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8000cd150 0x7fb8000d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.344+0000 7fb805a00640 1 -- 172.21.15.40:0/2333004686 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8000cc060 msgr2=0x7fb8000ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.344+0000 7fb805a00640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8000cc060 0x7fb8000ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.344+0000 7fb805a00640 1 -- 172.21.15.40:0/2333004686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8000cd150 msgr2=0x7fb8000d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.344+0000 7fb805a00640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8000cd150 0x7fb8000d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.344+0000 7fb805a00640 1 -- 172.21.15.40:0/2333004686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8000cafc0 con 0x7fb7e40051c0 2024-03-08T06:04:15.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.344+0000 7fb805a00640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 0x7fb80015c950 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0007f30 tx=0x7fb7f0011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:15.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.345+0000 7fb7fd600640 1 -- 172.21.15.40:0/2333004686 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f001a020 con 0x7fb7e40051c0 2024-03-08T06:04:15.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.345+0000 7fb7fd600640 1 -- 172.21.15.40:0/2333004686 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f0016470 con 0x7fb7e40051c0 2024-03-08T06:04:15.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.345+0000 7fb7fd600640 1 -- 172.21.15.40:0/2333004686 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0016860 con 0x7fb7e40051c0 2024-03-08T06:04:15.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.345+0000 7fb806e00640 1 -- 172.21.15.40:0/2333004686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8000cb2a0 con 0x7fb7e40051c0 2024-03-08T06:04:15.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.345+0000 7fb806e00640 1 -- 172.21.15.40:0/2333004686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8000cb860 con 0x7fb7e40051c0 2024-03-08T06:04:15.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.346+0000 7fb806e00640 1 -- 172.21.15.40:0/2333004686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb800075090 con 0x7fb7e40051c0 2024-03-08T06:04:15.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.347+0000 7fb7fd600640 1 -- 172.21.15.40:0/2333004686 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7f0005d10 con 0x7fb7e40051c0 2024-03-08T06:04:15.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.348+0000 7fb7fd600640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7e006b500 0x7fb7e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.348+0000 7fb7fd600640 1 -- 172.21.15.40:0/2333004686 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7f008b130 con 0x7fb7e40051c0 2024-03-08T06:04:15.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.348+0000 7fb805000640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7e006b500 0x7fb7e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.349+0000 7fb805000640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7e006b500 0x7fb7e006d9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000f60 tx=0x7fb7ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:15.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.353+0000 7fb7fd600640 1 -- 172.21.15.40:0/2333004686 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7f0024030 con 0x7fb7e40051c0 2024-03-08T06:04:15.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.553+0000 7fb806e00640 1 -- 172.21.15.40:0/2333004686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8000d17f0 con 0x7fb7e40051c0 2024-03-08T06:04:15.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.554+0000 7fb7fd600640 1 -- 172.21.15.40:0/2333004686 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7f0059f10 con 0x7fb7e40051c0 2024-03-08T06:04:15.555 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:15.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 -- 172.21.15.40:0/2333004686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7e006b500 msgr2=0x7fb7e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7e006b500 0x7fb7e006d9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000f60 tx=0x7fb7ec009040 comp rx=0 tx=0).stop 2024-03-08T06:04:15.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 -- 172.21.15.40:0/2333004686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 msgr2=0x7fb80015c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 0x7fb80015c950 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0007f30 tx=0x7fb7f0011660 comp rx=0 tx=0).stop 2024-03-08T06:04:15.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 -- 172.21.15.40:0/2333004686 shutdown_connections 2024-03-08T06:04:15.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7e006b500 0x7fb7e006d9c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8000cd150 0x7fb8000d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8000cc060 0x7fb8000ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 --2- 172.21.15.40:0/2333004686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e40051c0 0x7fb80015c950 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 -- 172.21.15.40:0/2333004686 >> 172.21.15.40:0/2333004686 conn(0x7fb80006c500 msgr2=0x7fb80006c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:15.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 -- 172.21.15.40:0/2333004686 shutdown_connections 2024-03-08T06:04:15.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.557+0000 7fb806e00640 1 -- 172.21.15.40:0/2333004686 wait complete. 2024-03-08T06:04:15.576 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.800+0000 7fb29ba00640 1 Processor -- start 2024-03-08T06:04:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.801+0000 7fb29ba00640 1 -- start start 2024-03-08T06:04:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.802+0000 7fb29ba00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb294071fa0 0x7fb2940cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.802+0000 7fb29ba00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29415ebd0 con 0x7fb294071bc0 2024-03-08T06:04:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.803+0000 7fb29ba00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29415ee70 con 0x7fb29415e760 2024-03-08T06:04:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.803+0000 7fb29ba00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29415f150 con 0x7fb294071fa0 2024-03-08T06:04:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.803+0000 7fb299000640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb294071bc0 0x7fb29415b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60982/0 (socket says 172.21.15.40:60982) 2024-03-08T06:04:15.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.803+0000 7fb299000640 1 -- 172.21.15.40:0/2669036438 learned_addr learned my addr 172.21.15.40:0/2669036438 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.803+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266195284 0 0) 0x7fb29415ebd0 con 0x7fb294071bc0 2024-03-08T06:04:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.803+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb27c005730 con 0x7fb294071bc0 2024-03-08T06:04:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.803+0000 7fb293e00640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb294071fa0 0x7fb2940cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1517522129 0 0) 0x7fb27c005730 con 0x7fb294071bc0 2024-03-08T06:04:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb294071fa0 msgr2=0x7fb2940cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb294071fa0 0x7fb2940cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 >> v1:172.21.15.53:6789/0 conn(0x7fb29415e760 legacy=0x7fb294144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29415f430 con 0x7fb294071bc0 2024-03-08T06:04:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fb288001690 con 0x7fb294071bc0 2024-03-08T06:04:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 >> v1:172.21.15.40:6789/0 conn(0x7fb294071bc0 legacy=0x7fb29415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb294071fa0 0x7fb27c005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb27c0061c0 0x7fb27c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 0x7fb27c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29415f150 con 0x7fb27c006f50 2024-03-08T06:04:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c00ddc0 con 0x7fb27c0061c0 2024-03-08T06:04:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c00dfc0 con 0x7fb294071fa0 2024-03-08T06:04:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.804+0000 7fb299a00640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 0x7fb27c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.805+0000 7fb293e00640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb27c0061c0 0x7fb27c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.805+0000 7fb299000640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb294071fa0 0x7fb27c005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.805+0000 7fb299a00640 1 -- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb294071fa0 msgr2=0x7fb27c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.805+0000 7fb299a00640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb294071fa0 0x7fb27c005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.805+0000 7fb299a00640 1 -- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb27c0061c0 msgr2=0x7fb27c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.805+0000 7fb299a00640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb27c0061c0 0x7fb27c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.805+0000 7fb299a00640 1 -- 172.21.15.40:0/2669036438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c005730 con 0x7fb27c006f50 2024-03-08T06:04:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.805+0000 7fb299000640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb294071fa0 0x7fb27c005bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.806+0000 7fb299a00640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 0x7fb27c0073d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb28c001e50 tx=0x7fb27c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42edeb0218093fd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.806+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c001a70 con 0x7fb27c006f50 2024-03-08T06:04:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.806+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c003b70 con 0x7fb27c006f50 2024-03-08T06:04:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.806+0000 7fb293400640 1 -- 172.21.15.40:0/2669036438 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0050d0 con 0x7fb27c006f50 2024-03-08T06:04:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.807+0000 7fb29ba00640 1 -- 172.21.15.40:0/2669036438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 msgr2=0x7fb27c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.807+0000 7fb29ba00640 1 --2- 172.21.15.40:0/2669036438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 0x7fb27c0073d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb28c001e50 tx=0x7fb27c007a40 comp rx=0 tx=0).stop 2024-03-08T06:04:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.807+0000 7fb299a00640 1 -- 172.21.15.40:0/2669036438 reap_dead start 2024-03-08T06:04:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.807+0000 7fb29ba00640 1 -- 172.21.15.40:0/2669036438 shutdown_connections 2024-03-08T06:04:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.807+0000 7fb29ba00640 1 -- 172.21.15.40:0/2669036438 >> 172.21.15.40:0/2669036438 conn(0x7fb29406ae30 msgr2=0x7fb29406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.807+0000 7fb29ba00640 1 -- 172.21.15.40:0/2669036438 shutdown_connections 2024-03-08T06:04:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.807+0000 7fb29ba00640 1 -- 172.21.15.40:0/2669036438 wait complete. 2024-03-08T06:04:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.809+0000 7fb29ba00640 1 Processor -- start 2024-03-08T06:04:15.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.809+0000 7fb29ba00640 1 -- start start 2024-03-08T06:04:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.810+0000 7fb29ba00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 0x7fb2941e7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.810+0000 7fb29ba00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb29415d0b0 0x7fb29415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.810+0000 7fb29ba00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2941e5920 0x7fb2940aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.810+0000 7fb29ba00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2940cf5b0 con 0x7fb27c006f50 2024-03-08T06:04:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.810+0000 7fb29ba00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb294144920 con 0x7fb2941e5920 2024-03-08T06:04:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.810+0000 7fb29ba00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2940cf730 con 0x7fb29415d0b0 2024-03-08T06:04:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.810+0000 7fb299000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 0x7fb2941e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.811+0000 7fb299000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 0x7fb2941e7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45980/0 (socket says 172.21.15.40:45980) 2024-03-08T06:04:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.811+0000 7fb299000640 1 -- 172.21.15.40:0/3570948638 learned_addr learned my addr 172.21.15.40:0/3570948638 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:15.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.811+0000 7fb293e00640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb29415d0b0 0x7fb29415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.811+0000 7fb299000640 1 -- 172.21.15.40:0/3570948638 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb29415d0b0 msgr2=0x7fb29415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.811+0000 7fb299000640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb29415d0b0 0x7fb29415ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.811+0000 7fb299a00640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2941e5920 0x7fb2940aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.811+0000 7fb299000640 1 -- 172.21.15.40:0/3570948638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2941e5920 msgr2=0x7fb2940aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:15.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.811+0000 7fb299000640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2941e5920 0x7fb2940aeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:15.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.811+0000 7fb299000640 1 -- 172.21.15.40:0/3570948638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2941e60e0 con 0x7fb27c006f50 2024-03-08T06:04:15.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.811+0000 7fb299000640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 0x7fb2941e7180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb28800d870 tx=0x7fb28800dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:15.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.812+0000 7fb290c00640 1 -- 172.21.15.40:0/3570948638 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288019070 con 0x7fb27c006f50 2024-03-08T06:04:15.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.812+0000 7fb290c00640 1 -- 172.21.15.40:0/3570948638 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2880074e0 con 0x7fb27c006f50 2024-03-08T06:04:15.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.812+0000 7fb290c00640 1 -- 172.21.15.40:0/3570948638 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2880049e0 con 0x7fb27c006f50 2024-03-08T06:04:15.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.812+0000 7fb29ba00640 1 -- 172.21.15.40:0/3570948638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2941e63c0 con 0x7fb27c006f50 2024-03-08T06:04:15.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.812+0000 7fb29ba00640 1 -- 172.21.15.40:0/3570948638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2941e6980 con 0x7fb27c006f50 2024-03-08T06:04:15.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.813+0000 7fb29ba00640 1 -- 172.21.15.40:0/3570948638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb294067cb0 con 0x7fb27c006f50 2024-03-08T06:04:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.814+0000 7fb290c00640 1 -- 172.21.15.40:0/3570948638 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb288004b80 con 0x7fb27c006f50 2024-03-08T06:04:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.815+0000 7fb290c00640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb27406b4d0 0x7fb27406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.815+0000 7fb290c00640 1 -- 172.21.15.40:0/3570948638 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb288089d20 con 0x7fb27c006f50 2024-03-08T06:04:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.815+0000 7fb293e00640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb27406b4d0 0x7fb27406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:15.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.816+0000 7fb293e00640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb27406b4d0 0x7fb27406d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb29406f170 tx=0x7fb280008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:15.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:15.819+0000 7fb290c00640 1 -- 172.21.15.40:0/3570948638 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb288058bc0 con 0x7fb27c006f50 2024-03-08T06:04:16.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.021+0000 7fb29ba00640 1 -- 172.21.15.40:0/3570948638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2940b2d90 con 0x7fb27c006f50 2024-03-08T06:04:16.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.023+0000 7fb290c00640 1 -- 172.21.15.40:0/3570948638 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2880584c0 con 0x7fb27c006f50 2024-03-08T06:04:16.023 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:16.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.025+0000 7fb29ba00640 1 -- 172.21.15.40:0/3570948638 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb27406b4d0 msgr2=0x7fb27406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.025+0000 7fb29ba00640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb27406b4d0 0x7fb27406d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb29406f170 tx=0x7fb280008040 comp rx=0 tx=0).stop 2024-03-08T06:04:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.025+0000 7fb29ba00640 1 -- 172.21.15.40:0/3570948638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 msgr2=0x7fb2941e7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.025+0000 7fb29ba00640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 0x7fb2941e7180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb28800d870 tx=0x7fb28800dd30 comp rx=0 tx=0).stop 2024-03-08T06:04:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.026+0000 7fb29ba00640 1 -- 172.21.15.40:0/3570948638 shutdown_connections 2024-03-08T06:04:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.026+0000 7fb29ba00640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb27406b4d0 0x7fb27406d990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.026+0000 7fb29ba00640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2941e5920 0x7fb2940aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.026+0000 7fb29ba00640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb29415d0b0 0x7fb29415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.026+0000 7fb29ba00640 1 --2- 172.21.15.40:0/3570948638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb27c006f50 0x7fb2941e7180 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.026+0000 7fb29ba00640 1 -- 172.21.15.40:0/3570948638 >> 172.21.15.40:0/3570948638 conn(0x7fb29406ae30 msgr2=0x7fb294143fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.026+0000 7fb29ba00640 1 -- 172.21.15.40:0/3570948638 shutdown_connections 2024-03-08T06:04:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.026+0000 7fb29ba00640 1 -- 172.21.15.40:0/3570948638 wait complete. 2024-03-08T06:04:16.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:04:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.267+0000 7feda0200640 1 Processor -- start 2024-03-08T06:04:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.267+0000 7feda0200640 1 -- start start 2024-03-08T06:04:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.268+0000 7feda0200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed980cd170 0x7fed980cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.268+0000 7feda0200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed980cd550 0x7fed9815af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.268+0000 7feda0200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fed9815ed40 0x7fed9806e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.268+0000 7feda0200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9815e8b0 con 0x7fed980cd550 2024-03-08T06:04:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.268+0000 7feda0200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9815e560 con 0x7fed9815ed40 2024-03-08T06:04:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.268+0000 7feda0200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9815b7c0 con 0x7fed980cd170 2024-03-08T06:04:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9ce00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed980cd550 0x7fed9815af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9ce00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed980cd550 0x7fed9815af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45988/0 (socket says 172.21.15.40:45988) 2024-03-08T06:04:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9ce00640 1 -- 172.21.15.40:0/3407080782 learned_addr learned my addr 172.21.15.40:0/3407080782 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9d800640 1 --2- 172.21.15.40:0/3407080782 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed980cd170 0x7fed980cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9e200640 1 --2- 172.21.15.40:0/3407080782 >> v2:172.21.15.53:3300/0 conn(0x7fed9815ed40 0x7fed9806e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9ce00640 1 -- 172.21.15.40:0/3407080782 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed980cd170 msgr2=0x7fed980cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9ce00640 1 --2- 172.21.15.40:0/3407080782 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed980cd170 0x7fed980cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9ce00640 1 -- 172.21.15.40:0/3407080782 >> v2:172.21.15.53:3300/0 conn(0x7fed9815ed40 msgr2=0x7fed9806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9ce00640 1 --2- 172.21.15.40:0/3407080782 >> v2:172.21.15.53:3300/0 conn(0x7fed9815ed40 0x7fed9806e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9ce00640 1 -- 172.21.15.40:0/3407080782 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9815baa0 con 0x7fed980cd550 2024-03-08T06:04:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9d800640 1 --2- 172.21.15.40:0/3407080782 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed980cd170 0x7fed980cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.269+0000 7fed9e200640 1 --2- 172.21.15.40:0/3407080782 >> v2:172.21.15.53:3300/0 conn(0x7fed9815ed40 0x7fed9806e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.270+0000 7fed9ce00640 1 --2- 172.21.15.40:0/3407080782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed980cd550 0x7fed9815af70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fed980ce360 tx=0x7fed8402b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2981294c177881d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.271+0000 7fed97e00640 1 -- 172.21.15.40:0/3407080782 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed840317f0 con 0x7fed980cd550 2024-03-08T06:04:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.271+0000 7fed97e00640 1 -- 172.21.15.40:0/3407080782 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed84037070 con 0x7fed980cd550 2024-03-08T06:04:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.271+0000 7fed97e00640 1 -- 172.21.15.40:0/3407080782 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84031bb0 con 0x7fed980cd550 2024-03-08T06:04:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.271+0000 7feda0200640 1 -- 172.21.15.40:0/3407080782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed980cd550 msgr2=0x7fed9815af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.271+0000 7feda0200640 1 --2- 172.21.15.40:0/3407080782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed980cd550 0x7fed9815af70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fed980ce360 tx=0x7fed8402b890 comp rx=0 tx=0).stop 2024-03-08T06:04:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.271+0000 7feda0200640 1 -- 172.21.15.40:0/3407080782 shutdown_connections 2024-03-08T06:04:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.271+0000 7feda0200640 1 --2- 172.21.15.40:0/3407080782 >> v2:172.21.15.53:3300/0 conn(0x7fed9815ed40 0x7fed9806e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.271+0000 7feda0200640 1 --2- 172.21.15.40:0/3407080782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed980cd550 0x7fed9815af70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.271+0000 7feda0200640 1 --2- 172.21.15.40:0/3407080782 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed980cd170 0x7fed980cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.271+0000 7feda0200640 1 -- 172.21.15.40:0/3407080782 >> 172.21.15.40:0/3407080782 conn(0x7fed9806ac50 msgr2=0x7fed9806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.271+0000 7feda0200640 1 -- 172.21.15.40:0/3407080782 shutdown_connections 2024-03-08T06:04:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.272+0000 7feda0200640 1 -- 172.21.15.40:0/3407080782 wait complete. 2024-03-08T06:04:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.273+0000 7feda0200640 1 Processor -- start 2024-03-08T06:04:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.273+0000 7feda0200640 1 -- start start 2024-03-08T06:04:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.274+0000 7feda0200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed980cd170 0x7fed98154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.274+0000 7feda0200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed980cd550 0x7fed981e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.274+0000 7feda0200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed9815ed40 0x7fed9813dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.274+0000 7feda0200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9815f1d0 con 0x7fed9815ed40 2024-03-08T06:04:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.274+0000 7feda0200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9815b4b0 con 0x7fed980cd550 2024-03-08T06:04:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.274+0000 7feda0200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9815f350 con 0x7fed980cd170 2024-03-08T06:04:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9e200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed9815ed40 0x7fed9813dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9e200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed9815ed40 0x7fed9813dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45998/0 (socket says 172.21.15.40:45998) 2024-03-08T06:04:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9e200640 1 -- 172.21.15.40:0/1489357081 learned_addr learned my addr 172.21.15.40:0/1489357081 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9ce00640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed980cd550 0x7fed981e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9e200640 1 -- 172.21.15.40:0/1489357081 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed980cd170 msgr2=0x7fed98154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9d800640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed980cd170 0x7fed98154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9e200640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed980cd170 0x7fed98154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9e200640 1 -- 172.21.15.40:0/1489357081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed980cd550 msgr2=0x7fed981e65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9e200640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed980cd550 0x7fed981e65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9e200640 1 -- 172.21.15.40:0/1489357081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed98152f40 con 0x7fed9815ed40 2024-03-08T06:04:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9ce00640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed980cd550 0x7fed981e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.275+0000 7fed9e200640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed9815ed40 0x7fed9813dcf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fed9000cef0 tx=0x7fed9000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.276+0000 7fed95600640 1 -- 172.21.15.40:0/1489357081 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9001a070 con 0x7fed9815ed40 2024-03-08T06:04:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.276+0000 7fed95600640 1 -- 172.21.15.40:0/1489357081 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9000d860 con 0x7fed9815ed40 2024-03-08T06:04:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.276+0000 7fed95600640 1 -- 172.21.15.40:0/1489357081 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000dc50 con 0x7fed9815ed40 2024-03-08T06:04:16.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.276+0000 7feda0200640 1 -- 172.21.15.40:0/1489357081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed98153220 con 0x7fed9815ed40 2024-03-08T06:04:16.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.276+0000 7feda0200640 1 -- 172.21.15.40:0/1489357081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed981537e0 con 0x7fed9815ed40 2024-03-08T06:04:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.277+0000 7feda0200640 1 -- 172.21.15.40:0/1489357081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed98067cb0 con 0x7fed9815ed40 2024-03-08T06:04:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.278+0000 7fed95600640 1 -- 172.21.15.40:0/1489357081 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed9001d070 con 0x7fed9815ed40 2024-03-08T06:04:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.279+0000 7fed95600640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fed7806b4d0 0x7fed7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.279+0000 7fed95600640 1 -- 172.21.15.40:0/1489357081 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed9008aa10 con 0x7fed9815ed40 2024-03-08T06:04:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.279+0000 7fed9d800640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fed7806b4d0 0x7fed7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.280+0000 7fed9d800640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fed7806b4d0 0x7fed7806d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00b840 tx=0x7fed8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:16.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.284+0000 7fed95600640 1 -- 172.21.15.40:0/1489357081 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed90059940 con 0x7fed9815ed40 2024-03-08T06:04:16.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.523+0000 7feda0200640 1 -- 172.21.15.40:0/1489357081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed980b4120 con 0x7fed9815ed40 2024-03-08T06:04:16.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.524+0000 7fed95600640 1 -- 172.21.15.40:0/1489357081 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fed90059240 con 0x7fed9815ed40 2024-03-08T06:04:16.525 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:16.525 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3266388900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3266388900},{"type":"v1","addr":"172.21.15.40:6837","nonce":3266388900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2471668569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2471668569},{"type":"v1","addr":"172.21.15.53:6834","nonce":2471668569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3262228631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3262228631},{"type":"v1","addr":"172.21.15.53:6835","nonce":3262228631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:04:02.003553+0000","modified":"2024-03-08T06:04:05.045865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3709744112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3709744112},{"type":"v1","addr":"172.21.15.40:6835","nonce":3709744112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:04:16.525 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-03-08T06:04:16.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.527+0000 7feda0200640 1 -- 172.21.15.40:0/1489357081 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fed7806b4d0 msgr2=0x7fed7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.527+0000 7feda0200640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fed7806b4d0 0x7fed7806d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00b840 tx=0x7fed8c008040 comp rx=0 tx=0).stop 2024-03-08T06:04:16.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.527+0000 7feda0200640 1 -- 172.21.15.40:0/1489357081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed9815ed40 msgr2=0x7fed9813dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.527+0000 7feda0200640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed9815ed40 0x7fed9813dcf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fed9000cef0 tx=0x7fed9000a710 comp rx=0 tx=0).stop 2024-03-08T06:04:16.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.527+0000 7feda0200640 1 -- 172.21.15.40:0/1489357081 shutdown_connections 2024-03-08T06:04:16.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.527+0000 7feda0200640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fed7806b4d0 0x7fed7806d990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.527+0000 7feda0200640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed9815ed40 0x7fed9813dcf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.527+0000 7feda0200640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed980cd550 0x7fed981e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.527+0000 7feda0200640 1 --2- 172.21.15.40:0/1489357081 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed980cd170 0x7fed98154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.527+0000 7feda0200640 1 -- 172.21.15.40:0/1489357081 >> 172.21.15.40:0/1489357081 conn(0x7fed9806ac50 msgr2=0x7fed980b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:16.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.527+0000 7feda0200640 1 -- 172.21.15.40:0/1489357081 shutdown_connections 2024-03-08T06:04:16.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.528+0000 7feda0200640 1 -- 172.21.15.40:0/1489357081 wait complete. 2024-03-08T06:04:16.549 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.759+0000 7fbe72200640 1 Processor -- start 2024-03-08T06:04:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.759+0000 7fbe72200640 1 -- start start 2024-03-08T06:04:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.761+0000 7fbe72200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbe6c15a770 0x7fbe6c15ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.761+0000 7fbe72200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe6c06ea40 0x7fbe6c15ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.761+0000 7fbe72200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe6c0cd020 con 0x7fbe6c0cc310 2024-03-08T06:04:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.761+0000 7fbe72200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6c0ccdd0 con 0x7fbe6c06ea40 2024-03-08T06:04:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.761+0000 7fbe72200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c15e8f0 con 0x7fbe6c15a770 2024-03-08T06:04:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.761+0000 7fbe6be00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbe6c0cc310 0x7fbe6c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60986/0 (socket says 172.21.15.40:60986) 2024-03-08T06:04:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.761+0000 7fbe6be00640 1 -- 172.21.15.40:0/467839823 learned_addr learned my addr 172.21.15.40:0/467839823 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.761+0000 7fbe71800640 1 --2- 172.21.15.40:0/467839823 >> v2:172.21.15.40:3300/0 conn(0x7fbe6c06ea40 0x7fbe6c15ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.762+0000 7fbe70e00640 1 --2- 172.21.15.40:0/467839823 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbe6c15a770 0x7fbe6c15ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.762+0000 7fbe6b400640 1 -- 172.21.15.40:0/467839823 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096585930 0 0) 0x7fbe6c0cd020 con 0x7fbe6c0cc310 2024-03-08T06:04:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.762+0000 7fbe71800640 1 -- 172.21.15.40:0/467839823 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbe6c15a770 msgr2=0x7fbe6c15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.762+0000 7fbe71800640 1 --2- 172.21.15.40:0/467839823 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbe6c15a770 0x7fbe6c15ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.762+0000 7fbe71800640 1 -- 172.21.15.40:0/467839823 >> v1:172.21.15.40:6789/0 conn(0x7fbe6c0cc310 legacy=0x7fbe6c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.762+0000 7fbe71800640 1 -- 172.21.15.40:0/467839823 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c0cd300 con 0x7fbe6c06ea40 2024-03-08T06:04:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.763+0000 7fbe71800640 1 --2- 172.21.15.40:0/467839823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe6c06ea40 0x7fbe6c15ab50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009d90 tx=0x7fbe6402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ed9016c567c6df server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.763+0000 7fbe6b400640 1 -- 172.21.15.40:0/467839823 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64039070 con 0x7fbe6c06ea40 2024-03-08T06:04:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.764+0000 7fbe6b400640 1 -- 172.21.15.40:0/467839823 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6402cc50 con 0x7fbe6c06ea40 2024-03-08T06:04:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.764+0000 7fbe6b400640 1 -- 172.21.15.40:0/467839823 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64034440 con 0x7fbe6c06ea40 2024-03-08T06:04:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.764+0000 7fbe72200640 1 -- 172.21.15.40:0/467839823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe6c06ea40 msgr2=0x7fbe6c15ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.764+0000 7fbe72200640 1 --2- 172.21.15.40:0/467839823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe6c06ea40 0x7fbe6c15ab50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009d90 tx=0x7fbe6402bea0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.764+0000 7fbe72200640 1 -- 172.21.15.40:0/467839823 shutdown_connections 2024-03-08T06:04:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.764+0000 7fbe72200640 1 --2- 172.21.15.40:0/467839823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe6c06ea40 0x7fbe6c15ab50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.764+0000 7fbe72200640 1 --2- 172.21.15.40:0/467839823 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbe6c15a770 0x7fbe6c15ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.764+0000 7fbe72200640 1 -- 172.21.15.40:0/467839823 >> 172.21.15.40:0/467839823 conn(0x7fbe6c0b1850 msgr2=0x7fbe6c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.764+0000 7fbe72200640 1 -- 172.21.15.40:0/467839823 shutdown_connections 2024-03-08T06:04:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.764+0000 7fbe72200640 1 -- 172.21.15.40:0/467839823 wait complete. 2024-03-08T06:04:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.766+0000 7fbe72200640 1 Processor -- start 2024-03-08T06:04:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.766+0000 7fbe72200640 1 -- start start 2024-03-08T06:04:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.767+0000 7fbe72200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbe6c06ea40 0x7fbe6c1e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.767+0000 7fbe72200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0cc310 0x7fbe6c1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.767+0000 7fbe72200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe6c15a770 0x7fbe6c1460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.767+0000 7fbe72200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0b4e00 con 0x7fbe6c15a770 2024-03-08T06:04:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.767+0000 7fbe72200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c15f2c0 con 0x7fbe6c0cc310 2024-03-08T06:04:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.767+0000 7fbe72200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0cc790 con 0x7fbe6c06ea40 2024-03-08T06:04:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.767+0000 7fbe71800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe6c15a770 0x7fbe6c1460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.767+0000 7fbe71800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe6c15a770 0x7fbe6c1460e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46020/0 (socket says 172.21.15.40:46020) 2024-03-08T06:04:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.767+0000 7fbe71800640 1 -- 172.21.15.40:0/690926168 learned_addr learned my addr 172.21.15.40:0/690926168 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.768+0000 7fbe6be00640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0cc310 0x7fbe6c1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.768+0000 7fbe70e00640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbe6c06ea40 0x7fbe6c1e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.768+0000 7fbe71800640 1 -- 172.21.15.40:0/690926168 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbe6c06ea40 msgr2=0x7fbe6c1e55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.768+0000 7fbe71800640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbe6c06ea40 0x7fbe6c1e55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.768+0000 7fbe71800640 1 -- 172.21.15.40:0/690926168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0cc310 msgr2=0x7fbe6c1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.768+0000 7fbe71800640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0cc310 0x7fbe6c1554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.768+0000 7fbe71800640 1 -- 172.21.15.40:0/690926168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c1e4530 con 0x7fbe6c15a770 2024-03-08T06:04:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.768+0000 7fbe71800640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe6c15a770 0x7fbe6c1460e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009d90 tx=0x7fbe64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.769+0000 7fbe68c00640 1 -- 172.21.15.40:0/690926168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64039050 con 0x7fbe6c15a770 2024-03-08T06:04:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.769+0000 7fbe68c00640 1 -- 172.21.15.40:0/690926168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe64045070 con 0x7fbe6c15a770 2024-03-08T06:04:16.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.769+0000 7fbe68c00640 1 -- 172.21.15.40:0/690926168 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6403e610 con 0x7fbe6c15a770 2024-03-08T06:04:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.769+0000 7fbe72200640 1 -- 172.21.15.40:0/690926168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6c1e4810 con 0x7fbe6c15a770 2024-03-08T06:04:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.769+0000 7fbe72200640 1 -- 172.21.15.40:0/690926168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6c1e4dd0 con 0x7fbe6c15a770 2024-03-08T06:04:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.770+0000 7fbe72200640 1 -- 172.21.15.40:0/690926168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6c075100 con 0x7fbe6c15a770 2024-03-08T06:04:16.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.771+0000 7fbe68c00640 1 -- 172.21.15.40:0/690926168 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe6403e7b0 con 0x7fbe6c15a770 2024-03-08T06:04:16.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.772+0000 7fbe68c00640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbe5006b4d0 0x7fbe5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:16.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.772+0000 7fbe68c00640 1 -- 172.21.15.40:0/690926168 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe640acd40 con 0x7fbe6c15a770 2024-03-08T06:04:16.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.772+0000 7fbe70e00640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbe5006b4d0 0x7fbe5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:16.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.773+0000 7fbe70e00640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbe5006b4d0 0x7fbe5006d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe60006fd0 tx=0x7fbe60008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:16.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.777+0000 7fbe68c00640 1 -- 172.21.15.40:0/690926168 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe6407bc50 con 0x7fbe6c15a770 2024-03-08T06:04:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.975+0000 7fbe72200640 1 -- 172.21.15.40:0/690926168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe6c153580 con 0x7fbe6c15a770 2024-03-08T06:04:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.978+0000 7fbe68c00640 1 -- 172.21.15.40:0/690926168 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7fbe6407b550 con 0x7fbe6c15a770 2024-03-08T06:04:16.979 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:16.980 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.981+0000 7fbe72200640 1 -- 172.21.15.40:0/690926168 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbe5006b4d0 msgr2=0x7fbe5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.981+0000 7fbe72200640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbe5006b4d0 0x7fbe5006d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe60006fd0 tx=0x7fbe60008040 comp rx=0 tx=0).stop 2024-03-08T06:04:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.981+0000 7fbe72200640 1 -- 172.21.15.40:0/690926168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe6c15a770 msgr2=0x7fbe6c1460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.981+0000 7fbe72200640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe6c15a770 0x7fbe6c1460e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009d90 tx=0x7fbe64004a40 comp rx=0 tx=0).stop 2024-03-08T06:04:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.981+0000 7fbe72200640 1 -- 172.21.15.40:0/690926168 shutdown_connections 2024-03-08T06:04:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.981+0000 7fbe72200640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbe5006b4d0 0x7fbe5006d990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.981+0000 7fbe72200640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe6c15a770 0x7fbe6c1460e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.981+0000 7fbe72200640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe6c0cc310 0x7fbe6c1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.981+0000 7fbe72200640 1 --2- 172.21.15.40:0/690926168 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbe6c06ea40 0x7fbe6c1e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.981+0000 7fbe72200640 1 -- 172.21.15.40:0/690926168 >> 172.21.15.40:0/690926168 conn(0x7fbe6c0b1850 msgr2=0x7fbe6c0cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.982+0000 7fbe72200640 1 -- 172.21.15.40:0/690926168 shutdown_connections 2024-03-08T06:04:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:16.982+0000 7fbe72200640 1 -- 172.21.15.40:0/690926168 wait complete. 2024-03-08T06:04:17.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:04:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.221+0000 7fa81b400640 1 Processor -- start 2024-03-08T06:04:17.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.221+0000 7fa81b400640 1 -- start start 2024-03-08T06:04:17.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.223+0000 7fa81b400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa8140d1800 0x7fa8140b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:17.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.223+0000 7fa81b400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa8140b5bf0 0x7fa8140b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:17.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.223+0000 7fa81b400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81415e100 con 0x7fa81415dd20 2024-03-08T06:04:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.223+0000 7fa81b400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8140d3a60 con 0x7fa8140b5bf0 2024-03-08T06:04:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.223+0000 7fa81b400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b6050 con 0x7fa8140d1800 2024-03-08T06:04:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.223+0000 7fa818a00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa81415dd20 0x7fa8140cec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38134/0 (socket says 172.21.15.40:38134) 2024-03-08T06:04:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.223+0000 7fa818a00640 1 -- 172.21.15.40:0/153658021 learned_addr learned my addr 172.21.15.40:0/153658021 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa819400640 1 --2- 172.21.15.40:0/153658021 >> v2:172.21.15.53:3300/0 conn(0x7fa8140b5bf0 0x7fa8140b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813e00640 1 --2- 172.21.15.40:0/153658021 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa8140d1800 0x7fa8140b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251781830 0 0) 0x7fa81415e100 con 0x7fa81415dd20 2024-03-08T06:04:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7fc003520 con 0x7fa81415dd20 2024-03-08T06:04:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2245079687 0 0) 0x7fa7fc003520 con 0x7fa81415dd20 2024-03-08T06:04:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa8140d1800 msgr2=0x7fa8140b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:17.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 --2- 172.21.15.40:0/153658021 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa8140d1800 0x7fa8140b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 >> v2:172.21.15.53:3300/0 conn(0x7fa8140b5bf0 msgr2=0x7fa8140b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 --2- 172.21.15.40:0/153658021 >> v2:172.21.15.53:3300/0 conn(0x7fa8140b5bf0 0x7fa8140b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140d29c0 con 0x7fa81415dd20 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fa808001670 con 0x7fa81415dd20 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 >> v1:172.21.15.40:6789/0 conn(0x7fa81415dd20 legacy=0x7fa8140cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 --2- 172.21.15.40:0/153658021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7fc0039b0 0x7fa7fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 --2- 172.21.15.40:0/153658021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 0x7fa7fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 --2- 172.21.15.40:0/153658021 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa8140d1800 0x7fa7fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140d3a60 con 0x7fa7fc004800 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc00b9c0 con 0x7fa7fc0039b0 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.224+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc00bb70 con 0x7fa8140d1800 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.225+0000 7fa813e00640 1 --2- 172.21.15.40:0/153658021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 0x7fa7fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:17.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.225+0000 7fa818a00640 1 --2- 172.21.15.40:0/153658021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7fc0039b0 0x7fa7fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.225+0000 7fa813e00640 1 -- 172.21.15.40:0/153658021 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa8140d1800 msgr2=0x7fa7fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.225+0000 7fa813e00640 1 --2- 172.21.15.40:0/153658021 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa8140d1800 0x7fa7fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.225+0000 7fa813e00640 1 -- 172.21.15.40:0/153658021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7fc0039b0 msgr2=0x7fa7fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.225+0000 7fa813e00640 1 --2- 172.21.15.40:0/153658021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7fc0039b0 0x7fa7fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:17.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.225+0000 7fa813e00640 1 -- 172.21.15.40:0/153658021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc003520 con 0x7fa7fc004800 2024-03-08T06:04:17.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.226+0000 7fa813e00640 1 --2- 172.21.15.40:0/153658021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 0x7fa7fc004c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa800009ac0 tx=0x7fa7fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bb8ce66b540a78 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:17.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.226+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800039070 con 0x7fa7fc004800 2024-03-08T06:04:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.226+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80002c710 con 0x7fa7fc004800 2024-03-08T06:04:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.226+0000 7fa813400640 1 -- 172.21.15.40:0/153658021 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800009690 con 0x7fa7fc004800 2024-03-08T06:04:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.227+0000 7fa81b400640 1 -- 172.21.15.40:0/153658021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 msgr2=0x7fa7fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.227+0000 7fa81b400640 1 --2- 172.21.15.40:0/153658021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 0x7fa7fc004c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa800009ac0 tx=0x7fa7fc005830 comp rx=0 tx=0).stop 2024-03-08T06:04:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.227+0000 7fa819400640 1 -- 172.21.15.40:0/153658021 reap_dead start 2024-03-08T06:04:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.227+0000 7fa81b400640 1 -- 172.21.15.40:0/153658021 shutdown_connections 2024-03-08T06:04:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.227+0000 7fa81b400640 1 -- 172.21.15.40:0/153658021 >> 172.21.15.40:0/153658021 conn(0x7fa814067240 msgr2=0x7fa8140d06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.227+0000 7fa81b400640 1 -- 172.21.15.40:0/153658021 shutdown_connections 2024-03-08T06:04:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.227+0000 7fa81b400640 1 -- 172.21.15.40:0/153658021 wait complete. 2024-03-08T06:04:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.229+0000 7fa81b400640 1 Processor -- start 2024-03-08T06:04:17.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.229+0000 7fa81b400640 1 -- start start 2024-03-08T06:04:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.230+0000 7fa81b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 0x7fa8141f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.230+0000 7fa81b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa814156f40 0x7fa814152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.230+0000 7fa81b400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa814160b40 0x7fa8140cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.230+0000 7fa81b400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140d2f80 con 0x7fa7fc004800 2024-03-08T06:04:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.230+0000 7fa81b400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140d35d0 con 0x7fa814156f40 2024-03-08T06:04:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.230+0000 7fa81b400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140d3260 con 0x7fa814160b40 2024-03-08T06:04:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.230+0000 7fa818a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 0x7fa8141f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.230+0000 7fa818a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 0x7fa8141f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56104/0 (socket says 172.21.15.40:56104) 2024-03-08T06:04:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.230+0000 7fa818a00640 1 -- 172.21.15.40:0/378642025 learned_addr learned my addr 172.21.15.40:0/378642025 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa813e00640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa814156f40 0x7fa814152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa819400640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa814160b40 0x7fa8140cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa818a00640 1 -- 172.21.15.40:0/378642025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa814160b40 msgr2=0x7fa8140cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa818a00640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa814160b40 0x7fa8140cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa818a00640 1 -- 172.21.15.40:0/378642025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa814156f40 msgr2=0x7fa814152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa818a00640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa814156f40 0x7fa814152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa818a00640 1 -- 172.21.15.40:0/378642025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81415f840 con 0x7fa7fc004800 2024-03-08T06:04:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa818a00640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 0x7fa8141f39a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa80800e7f0 tx=0x7fa80800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa810c00640 1 -- 172.21.15.40:0/378642025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80800eea0 con 0x7fa7fc004800 2024-03-08T06:04:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa810c00640 1 -- 172.21.15.40:0/378642025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa808018070 con 0x7fa7fc004800 2024-03-08T06:04:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa81b400640 1 -- 172.21.15.40:0/378642025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa814154370 con 0x7fa7fc004800 2024-03-08T06:04:17.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.231+0000 7fa810c00640 1 -- 172.21.15.40:0/378642025 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80800cab0 con 0x7fa7fc004800 2024-03-08T06:04:17.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.232+0000 7fa81b400640 1 -- 172.21.15.40:0/378642025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa814154610 con 0x7fa7fc004800 2024-03-08T06:04:17.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.233+0000 7fa81b400640 1 -- 172.21.15.40:0/378642025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa814067cb0 con 0x7fa7fc004800 2024-03-08T06:04:17.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.234+0000 7fa810c00640 1 -- 172.21.15.40:0/378642025 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa80800cc50 con 0x7fa7fc004800 2024-03-08T06:04:17.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.235+0000 7fa810c00640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7f406b4d0 0x7fa7f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:17.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.235+0000 7fa810c00640 1 -- 172.21.15.40:0/378642025 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa80808a220 con 0x7fa7fc004800 2024-03-08T06:04:17.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.235+0000 7fa813e00640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7f406b4d0 0x7fa7f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:17.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.236+0000 7fa813e00640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7f406b4d0 0x7fa7f406d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa800036000 tx=0x7fa800030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:17.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.240+0000 7fa810c00640 1 -- 172.21.15.40:0/378642025 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa808058f70 con 0x7fa7fc004800 2024-03-08T06:04:17.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.480+0000 7fa81b400640 1 -- 172.21.15.40:0/378642025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8141547e0 con 0x7fa7fc004800 2024-03-08T06:04:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.482+0000 7fa810c00640 1 -- 172.21.15.40:0/378642025 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fa808058870 con 0x7fa7fc004800 2024-03-08T06:04:17.482 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:17.482 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3266388900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3266388900},{"type":"v1","addr":"172.21.15.40:6837","nonce":3266388900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2471668569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2471668569},{"type":"v1","addr":"172.21.15.53:6834","nonce":2471668569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3262228631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3262228631},{"type":"v1","addr":"172.21.15.53:6835","nonce":3262228631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:04:02.003553+0000","modified":"2024-03-08T06:04:05.045865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3709744112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3709744112},{"type":"v1","addr":"172.21.15.40:6835","nonce":3709744112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:04:17.482 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-03-08T06:04:17.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.484+0000 7fa81b400640 1 -- 172.21.15.40:0/378642025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7f406b4d0 msgr2=0x7fa7f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:17.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.484+0000 7fa81b400640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7f406b4d0 0x7fa7f406d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa800036000 tx=0x7fa800030000 comp rx=0 tx=0).stop 2024-03-08T06:04:17.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.484+0000 7fa81b400640 1 -- 172.21.15.40:0/378642025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 msgr2=0x7fa8141f39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:17.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.484+0000 7fa81b400640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 0x7fa8141f39a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa80800e7f0 tx=0x7fa80800ecb0 comp rx=0 tx=0).stop 2024-03-08T06:04:17.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.484+0000 7fa81b400640 1 -- 172.21.15.40:0/378642025 shutdown_connections 2024-03-08T06:04:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.484+0000 7fa81b400640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7f406b4d0 0x7fa7f406d990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.484+0000 7fa81b400640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa814160b40 0x7fa8140cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.484+0000 7fa81b400640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa814156f40 0x7fa814152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.484+0000 7fa81b400640 1 --2- 172.21.15.40:0/378642025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7fc004800 0x7fa8141f39a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.484+0000 7fa81b400640 1 -- 172.21.15.40:0/378642025 >> 172.21.15.40:0/378642025 conn(0x7fa814067240 msgr2=0x7fa8140b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.485+0000 7fa81b400640 1 -- 172.21.15.40:0/378642025 shutdown_connections 2024-03-08T06:04:17.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:17.485+0000 7fa81b400640 1 -- 172.21.15.40:0/378642025 wait complete. 2024-03-08T06:04:17.908 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-08T06:04:02.003553+0000', 'modified': '2024-03-08T06:04:05.045865+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4260}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4260': {'gid': 4260, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/3709744112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 3709744112}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3709744112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-08T06:04:17.908 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-08T06:04:17.908 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-08T06:04:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.136+0000 7f0c15200640 1 Processor -- start 2024-03-08T06:04:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.136+0000 7f0c15200640 1 -- start start 2024-03-08T06:04:18.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.138+0000 7f0c15200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c100aee90 0x7f0c101631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.138+0000 7f0c15200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1006d500 con 0x7f0c10071b10 2024-03-08T06:04:18.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.138+0000 7f0c15200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1006d7e0 con 0x7f0c100b44c0 2024-03-08T06:04:18.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.138+0000 7f0c15200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1006de30 con 0x7f0c100aee90 2024-03-08T06:04:18.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.139+0000 7f0c0fe00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0c10071b10 0x7f0c100b2890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38136/0 (socket says 172.21.15.40:38136) 2024-03-08T06:04:18.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.139+0000 7f0c0fe00640 1 -- 172.21.15.40:0/2517694891 learned_addr learned my addr 172.21.15.40:0/2517694891 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:18.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.139+0000 7f0c0f400640 1 --2- 172.21.15.40:0/2517694891 >> v2:172.21.15.53:3300/0 conn(0x7f0c100aee90 0x7f0c101631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.139+0000 7f0c0ea00640 1 -- 172.21.15.40:0/2517694891 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022520500 0 0) 0x7f0c1006d500 con 0x7f0c10071b10 2024-03-08T06:04:18.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.139+0000 7f0c0ea00640 1 -- 172.21.15.40:0/2517694891 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bf4003520 con 0x7f0c10071b10 2024-03-08T06:04:18.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.140+0000 7f0c0f400640 1 -- 172.21.15.40:0/2517694891 >> v1:172.21.15.53:6789/0 conn(0x7f0c100b44c0 legacy=0x7f0c10071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:18.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.140+0000 7f0c0f400640 1 -- 172.21.15.40:0/2517694891 >> v1:172.21.15.40:6789/0 conn(0x7f0c10071b10 legacy=0x7f0c100b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:18.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.140+0000 7f0c0f400640 1 -- 172.21.15.40:0/2517694891 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1006dac0 con 0x7f0c100aee90 2024-03-08T06:04:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.141+0000 7f0c0f400640 1 --2- 172.21.15.40:0/2517694891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c100aee90 0x7f0c101631c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097a0 tx=0x7f0c0002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8dfc66bb33c0e2b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.141+0000 7f0c0ea00640 1 -- 172.21.15.40:0/2517694891 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00034070 con 0x7f0c100aee90 2024-03-08T06:04:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.141+0000 7f0c0ea00640 1 -- 172.21.15.40:0/2517694891 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c00030610 con 0x7f0c100aee90 2024-03-08T06:04:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.141+0000 7f0c0ea00640 1 -- 172.21.15.40:0/2517694891 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c000309e0 con 0x7f0c100aee90 2024-03-08T06:04:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.142+0000 7f0c15200640 1 -- 172.21.15.40:0/2517694891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c100aee90 msgr2=0x7f0c101631c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.142+0000 7f0c15200640 1 --2- 172.21.15.40:0/2517694891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c100aee90 0x7f0c101631c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097a0 tx=0x7f0c0002b960 comp rx=0 tx=0).stop 2024-03-08T06:04:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.142+0000 7f0c15200640 1 -- 172.21.15.40:0/2517694891 shutdown_connections 2024-03-08T06:04:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.142+0000 7f0c15200640 1 --2- 172.21.15.40:0/2517694891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c100aee90 0x7f0c101631c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.142+0000 7f0c15200640 1 -- 172.21.15.40:0/2517694891 >> 172.21.15.40:0/2517694891 conn(0x7f0c100b1590 msgr2=0x7f0c100b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.142+0000 7f0c15200640 1 -- 172.21.15.40:0/2517694891 shutdown_connections 2024-03-08T06:04:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.142+0000 7f0c15200640 1 -- 172.21.15.40:0/2517694891 wait complete. 2024-03-08T06:04:18.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.144+0000 7f0c15200640 1 Processor -- start 2024-03-08T06:04:18.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.144+0000 7f0c15200640 1 -- start start 2024-03-08T06:04:18.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.145+0000 7f0c15200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10071b10 0x7f0c100adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.145+0000 7f0c15200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0c100aee90 0x7f0c101eb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.145+0000 7f0c15200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c100b44c0 0x7f0c1014e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.145+0000 7f0c15200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1006e2c0 con 0x7f0c100b44c0 2024-03-08T06:04:18.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.145+0000 7f0c15200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c100b4d10 con 0x7f0c10071b10 2024-03-08T06:04:18.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.145+0000 7f0c15200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1006e440 con 0x7f0c100aee90 2024-03-08T06:04:18.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.145+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c100b44c0 0x7f0c1014e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.145+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c100b44c0 0x7f0c1014e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56108/0 (socket says 172.21.15.40:56108) 2024-03-08T06:04:18.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.145+0000 7f0c0fe00640 1 -- 172.21.15.40:0/4276038349 learned_addr learned my addr 172.21.15.40:0/4276038349 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:18.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.146+0000 7f0c0f400640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10071b10 0x7f0c100adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.146+0000 7f0c06a00640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0c100aee90 0x7f0c101eb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.146+0000 7f0c0fe00640 1 -- 172.21.15.40:0/4276038349 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0c100aee90 msgr2=0x7f0c101eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:18.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.146+0000 7f0c0fe00640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0c100aee90 0x7f0c101eb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.146+0000 7f0c0fe00640 1 -- 172.21.15.40:0/4276038349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10071b10 msgr2=0x7f0c100adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:18.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.146+0000 7f0c0fe00640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10071b10 0x7f0c100adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.146+0000 7f0c0fe00640 1 -- 172.21.15.40:0/4276038349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c101ebb00 con 0x7f0c100b44c0 2024-03-08T06:04:18.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.146+0000 7f0c0fe00640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c100b44c0 0x7f0c1014e7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800cbb0 tx=0x7f0c0800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:18.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.146+0000 7f0c07e00640 1 -- 172.21.15.40:0/4276038349 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08017070 con 0x7f0c100b44c0 2024-03-08T06:04:18.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.147+0000 7f0c07e00640 1 -- 172.21.15.40:0/4276038349 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0800ad40 con 0x7f0c100b44c0 2024-03-08T06:04:18.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.147+0000 7f0c15200640 1 -- 172.21.15.40:0/4276038349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c101ebde0 con 0x7f0c100b44c0 2024-03-08T06:04:18.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.147+0000 7f0c07e00640 1 -- 172.21.15.40:0/4276038349 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08012600 con 0x7f0c100b44c0 2024-03-08T06:04:18.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.147+0000 7f0c15200640 1 -- 172.21.15.40:0/4276038349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1014c7b0 con 0x7f0c100b44c0 2024-03-08T06:04:18.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.149+0000 7f0c07e00640 1 -- 172.21.15.40:0/4276038349 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c0801a070 con 0x7f0c100b44c0 2024-03-08T06:04:18.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.150+0000 7f0c07e00640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0bec06b4d0 0x7f0bec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.150+0000 7f0c07e00640 1 -- 172.21.15.40:0/4276038349 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c08089f40 con 0x7f0c100b44c0 2024-03-08T06:04:18.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.150+0000 7f0c0f400640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0bec06b4d0 0x7f0bec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.151+0000 7f0c193ff740 1 Processor -- start 2024-03-08T06:04:18.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.151+0000 7f0c0f400640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0bec06b4d0 0x7f0bec06d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c00036000 tx=0x7f0c0003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:18.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.151+0000 7f0c193ff740 1 -- start start 2024-03-08T06:04:18.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4bd9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c09720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4bd9e70 0x563cc4c09c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c193ff740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc4b43600 con 0x563cc4c06d70 2024-03-08T06:04:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c193ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc4aec930 con 0x563cc4c06990 2024-03-08T06:04:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c193ff740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cc4b3f6a0 con 0x563cc4bd9e70 2024-03-08T06:04:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c06a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c09720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c06a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c09720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56124/0 (socket says 172.21.15.40:56124) 2024-03-08T06:04:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c06a00640 1 -- 172.21.15.40:0/3880953775 learned_addr learned my addr 172.21.15.40:0/3880953775 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c0f400640 1 --2- 172.21.15.40:0/3880953775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4bd9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c0fe00640 1 --2- 172.21.15.40:0/3880953775 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4bd9e70 0x563cc4c09c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c06a00640 1 -- 172.21.15.40:0/3880953775 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4bd9e70 msgr2=0x563cc4c09c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c06a00640 1 --2- 172.21.15.40:0/3880953775 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4bd9e70 0x563cc4c09c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c06a00640 1 -- 172.21.15.40:0/3880953775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 msgr2=0x563cc4bd9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c06a00640 1 --2- 172.21.15.40:0/3880953775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4bd9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c06a00640 1 -- 172.21.15.40:0/3880953775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cc48582d0 con 0x563cc4c06d70 2024-03-08T06:04:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.152+0000 7f0c0f400640 1 --2- 172.21.15.40:0/3880953775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4bd9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.153+0000 7f0c06a00640 1 --2- 172.21.15.40:0/3880953775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c09720 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8001520 tx=0x7f0bf80019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addc9abfa4f77962 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.153+0000 7f0bffe00640 1 -- 172.21.15.40:0/3880953775 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8019070 con 0x563cc4c06d70 2024-03-08T06:04:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.153+0000 7f0bffe00640 1 -- 172.21.15.40:0/3880953775 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf80052e0 con 0x563cc4c06d70 2024-03-08T06:04:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.153+0000 7f0bffe00640 1 -- 172.21.15.40:0/3880953775 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80037f0 con 0x563cc4c06d70 2024-03-08T06:04:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.154+0000 7f0c193ff740 1 -- 172.21.15.40:0/3880953775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 msgr2=0x563cc4c09720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.154+0000 7f0c193ff740 1 --2- 172.21.15.40:0/3880953775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c09720 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8001520 tx=0x7f0bf80019e0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.154+0000 7f0c193ff740 1 -- 172.21.15.40:0/3880953775 shutdown_connections 2024-03-08T06:04:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.154+0000 7f0c193ff740 1 --2- 172.21.15.40:0/3880953775 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4bd9e70 0x563cc4c09c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.154+0000 7f0c193ff740 1 --2- 172.21.15.40:0/3880953775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c09720 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.154+0000 7f0c193ff740 1 --2- 172.21.15.40:0/3880953775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4bd9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.154+0000 7f0c193ff740 1 -- 172.21.15.40:0/3880953775 >> 172.21.15.40:0/3880953775 conn(0x563cc4bdbba0 msgr2=0x563cc4c24720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.154+0000 7f0c193ff740 1 -- 172.21.15.40:0/3880953775 shutdown_connections 2024-03-08T06:04:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.154+0000 7f0c193ff740 1 -- 172.21.15.40:0/3880953775 wait complete. 2024-03-08T06:04:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.155+0000 7f0c193ff740 1 -- start start 2024-03-08T06:04:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.155+0000 7f0c193ff740 1 Processor -- start 2024-03-08T06:04:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4bf49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c1b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c1b780 0x563cc4c0bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c193ff740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc4bde140 con 0x563cc4c06d70 2024-03-08T06:04:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c193ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc4affd90 con 0x563cc4c06990 2024-03-08T06:04:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c193ff740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cc4bdd530 con 0x563cc4c1b780 2024-03-08T06:04:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c193ff740 10 client.0 _ll_register_callbacks cb 0x563cc4b3b350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c06a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c1b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c06a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c1b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56138/0 (socket says 172.21.15.40:56138) 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c06a00640 1 -- 172.21.15.40:0/1358879670 learned_addr learned my addr 172.21.15.40:0/1358879670 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c0f400640 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4bf49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c06a00640 1 -- 172.21.15.40:0/1358879670 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c1b780 msgr2=0x563cc4c0bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c06a00640 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c1b780 0x563cc4c0bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c06a00640 1 -- 172.21.15.40:0/1358879670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 msgr2=0x563cc4bf49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c06a00640 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4bf49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.156+0000 7f0c06a00640 1 -- 172.21.15.40:0/1358879670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cc49226b0 con 0x563cc4c06d70 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.157+0000 7f0c06a00640 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c1b240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80038d0 tx=0x7f0bf8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c9f51302ccff5a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.157+0000 7f0bfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-03-08T06:04:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.157+0000 7f0bfcc00640 1 -- 172.21.15.40:0/1358879670 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8023060 con 0x563cc4c06d70 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.157+0000 7f0bfcc00640 1 -- 172.21.15.40:0/1358879670 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bf800e490 con 0x563cc4c06d70 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.157+0000 7f0c193ff740 1 -- 172.21.15.40:0/1358879670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bf80052e0 con 0x563cc4c06d70 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.157+0000 7f0bfcc00640 1 -- 172.21.15.40:0/1358879670 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf8029050 con 0x563cc4c06d70 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.157+0000 7f0c193ff740 10 client.4605 fetch_fsmap learned FSMap version 8 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.157+0000 7f0c193ff740 1 -- 172.21.15.40:0/1358879670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x563cc4bf5510 con 0x563cc4c06d70 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.158+0000 7f0bfcc00640 1 -- 172.21.15.40:0/1358879670 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf80052e0 con 0x563cc4c06d70 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.158+0000 7f0bfcc00640 1 -- 172.21.15.40:0/1358879670 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0bf802c020 con 0x563cc4c06d70 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.158+0000 7f0c193ff740 10 client.4605 fetch_fsmap finished waiting for FSMap version 8 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.158+0000 7f0c193ff740 10 client.4605 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.158+0000 7f0c193ff740 20 client.4605 populate_metadata read hostname 'smithi040' 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.158+0000 7f0c193ff740 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.40:6834/3709744112,v1:172.21.15.40:6835/3709744112] conn(0x563cc4c1f3a0 0x563cc4bf6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.158+0000 7f0c193ff740 4 client.4605 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T06:04:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.158+0000 7f0c193ff740 1 -- 172.21.15.40:0/1358879670 --> [v2:172.21.15.40:6834/3709744112,v1:172.21.15.40:6835/3709744112] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cc4b50b40 con 0x563cc4c1f3a0 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.158+0000 7f0c0f400640 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.40:6834/3709744112,v1:172.21.15.40:6835/3709744112] conn(0x563cc4c1f3a0 0x563cc4bf6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.159+0000 7f0c0f400640 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.40:6834/3709744112,v1:172.21.15.40:6835/3709744112] conn(0x563cc4c1f3a0 0x563cc4bf6080 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cda92b62841d556b server_cookie=660056d9a88c183e in_seq=0 out_seq=0 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.159+0000 7f0bfcc00640 10 client.4605 ms_handle_connect on v2:172.21.15.40:6834/3709744112 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.159+0000 7f0bfcc00640 1 -- 172.21.15.40:0/1358879670 <== mds.0 v2:172.21.15.40:6834/3709744112 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563cc4b50b40 con 0x563cc4c1f3a0 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.159+0000 7f0bfcc00640 10 client.4605 handle_command_reply: tid=0 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 1 client.4605 shutdown 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 1 -- 172.21.15.40:0/1358879670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 msgr2=0x563cc4c1b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c1b240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80038d0 tx=0x7f0bf8000ed0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 1 -- 172.21.15.40:0/1358879670 shutdown_connections 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.40:6834/3709744112,v1:172.21.15.40:6835/3709744112] conn(0x563cc4c1f3a0 0x563cc4bf6080 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c1b780 0x563cc4c0bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4c1b240 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 1 --2- 172.21.15.40:0/1358879670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4bf49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 1 -- 172.21.15.40:0/1358879670 >> 172.21.15.40:0/1358879670 conn(0x563cc4bd9e70 msgr2=0x563cc4c09710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0bfcc00640 0 client.4605 ms_handle_reset on v2:172.21.15.40:6834/3709744112 2024-03-08T06:04:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 1 -- 172.21.15.40:0/1358879670 shutdown_connections 2024-03-08T06:04:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 1 -- 172.21.15.40:0/1358879670 wait complete. 2024-03-08T06:04:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.160+0000 7f0c193ff740 20 client.4605 trim_cache size 0 max 16384 2024-03-08T06:04:18.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.177+0000 7f0c193ff740 1 Processor -- start 2024-03-08T06:04:18.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.178+0000 7f0c193ff740 1 -- start start 2024-03-08T06:04:18.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.178+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4c37050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.178+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bf2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.178+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c1b780 0x563cc4bf2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.178+0000 7f0c193ff740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1014e1a0 con 0x563cc4c06d70 2024-03-08T06:04:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.178+0000 7f0c193ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c101fbf60 con 0x563cc4c06990 2024-03-08T06:04:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.178+0000 7f0c193ff740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1014dba0 con 0x563cc4c1b780 2024-03-08T06:04:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.179+0000 7f0c06a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bf2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.179+0000 7f0c06a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bf2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56152/0 (socket says 172.21.15.40:56152) 2024-03-08T06:04:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.179+0000 7f0c06a00640 1 -- 172.21.15.40:0/892673966 learned_addr learned my addr 172.21.15.40:0/892673966 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.179+0000 7f0c06a00640 1 -- 172.21.15.40:0/892673966 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c1b780 msgr2=0x563cc4bf2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.179+0000 7f0c06a00640 1 --2- 172.21.15.40:0/892673966 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c1b780 0x563cc4bf2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.179+0000 7f0c06a00640 1 -- 172.21.15.40:0/892673966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 msgr2=0x563cc4c37050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:18.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.179+0000 7f0c06a00640 1 --2- 172.21.15.40:0/892673966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4c37050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.179+0000 7f0c06a00640 1 -- 172.21.15.40:0/892673966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cc4bd8440 con 0x563cc4c06d70 2024-03-08T06:04:18.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.179+0000 7f0c06a00640 1 --2- 172.21.15.40:0/892673966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bf2200 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8001240 tx=0x7f0bf801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e685b4588df329ad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:18.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.179+0000 7f0bff400640 1 -- 172.21.15.40:0/892673966 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8019070 con 0x563cc4c06d70 2024-03-08T06:04:18.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.180+0000 7f0bff400640 1 -- 172.21.15.40:0/892673966 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf8031020 con 0x563cc4c06d70 2024-03-08T06:04:18.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.180+0000 7f0bff400640 1 -- 172.21.15.40:0/892673966 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800e490 con 0x563cc4c06d70 2024-03-08T06:04:18.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.180+0000 7f0c193ff740 1 -- 172.21.15.40:0/892673966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 msgr2=0x563cc4bf2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:18.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.180+0000 7f0c193ff740 1 --2- 172.21.15.40:0/892673966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bf2200 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8001240 tx=0x7f0bf801c000 comp rx=0 tx=0).stop 2024-03-08T06:04:18.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.180+0000 7f0c193ff740 1 -- 172.21.15.40:0/892673966 shutdown_connections 2024-03-08T06:04:18.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.180+0000 7f0c193ff740 1 --2- 172.21.15.40:0/892673966 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c1b780 0x563cc4bf2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.180+0000 7f0c193ff740 1 --2- 172.21.15.40:0/892673966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bf2200 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.180+0000 7f0c193ff740 1 --2- 172.21.15.40:0/892673966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c06990 0x563cc4c37050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.180+0000 7f0c193ff740 1 -- 172.21.15.40:0/892673966 >> 172.21.15.40:0/892673966 conn(0x563cc4bd9e70 msgr2=0x563cc4c09160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:18.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.180+0000 7f0c193ff740 1 -- 172.21.15.40:0/892673966 shutdown_connections 2024-03-08T06:04:18.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.180+0000 7f0c193ff740 1 -- 172.21.15.40:0/892673966 wait complete. 2024-03-08T06:04:18.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.181+0000 7f0c193ff740 1 -- start start 2024-03-08T06:04:18.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.181+0000 7f0c193ff740 1 Processor -- start 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.182+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c06990 0x563cc4c1e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.182+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bde860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.182+0000 7f0c193ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c1b780 0x563cc4bdeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.182+0000 7f0c193ff740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc4c1f220 con 0x563cc4c06d70 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.182+0000 7f0c193ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc4bf6ab0 con 0x563cc4c1b780 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.182+0000 7f0c193ff740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cc4c1f7b0 con 0x563cc4c06990 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.182+0000 7f0c193ff740 10 client.0 _ll_register_callbacks cb 0x7f0bf8007e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.182+0000 7f0c06a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bde860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.182+0000 7f0c06a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bde860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56158/0 (socket says 172.21.15.40:56158) 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.182+0000 7f0c06a00640 1 -- 172.21.15.40:0/393714267 learned_addr learned my addr 172.21.15.40:0/393714267 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0c0f400640 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c06990 0x563cc4c1e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0c06a00640 1 -- 172.21.15.40:0/393714267 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c06990 msgr2=0x563cc4c1e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0c06a00640 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c06990 0x563cc4c1e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0c0fe00640 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c1b780 0x563cc4bdeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0c06a00640 1 -- 172.21.15.40:0/393714267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c1b780 msgr2=0x563cc4bdeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0c06a00640 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c1b780 0x563cc4bdeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0c06a00640 1 -- 172.21.15.40:0/393714267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cc4c47230 con 0x563cc4c06d70 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0c06a00640 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bde860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e160 tx=0x7f0bf8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ba8d283c2affd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0bfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0bfcc00640 1 -- 172.21.15.40:0/393714267 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8007040 con 0x563cc4c06d70 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0bfcc00640 1 -- 172.21.15.40:0/393714267 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bf80071e0 con 0x563cc4c06d70 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.183+0000 7f0c193ff740 1 -- 172.21.15.40:0/393714267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bf8031020 con 0x563cc4c06d70 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0bfcc00640 1 -- 172.21.15.40:0/393714267 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf8044050 con 0x563cc4c06d70 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0c193ff740 10 client.4611 fetch_fsmap learned FSMap version 8 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0c193ff740 1 -- 172.21.15.40:0/393714267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x563cc4bf41d0 con 0x563cc4c06d70 2024-03-08T06:04:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0bfcc00640 1 -- 172.21.15.40:0/393714267 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf8031020 con 0x563cc4c06d70 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0bfcc00640 1 -- 172.21.15.40:0/393714267 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0bf8047020 con 0x563cc4c06d70 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0c193ff740 10 client.4611 fetch_fsmap finished waiting for FSMap version 8 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0c193ff740 10 client.4611 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0c193ff740 20 client.4611 populate_metadata read hostname 'smithi040' 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0c193ff740 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.40:6834/3709744112,v1:172.21.15.40:6835/3709744112] conn(0x563cc4c25b50 0x563cc4c27ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0c193ff740 4 client.4611 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0c193ff740 1 -- 172.21.15.40:0/393714267 --> [v2:172.21.15.40:6834/3709744112,v1:172.21.15.40:6835/3709744112] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563cc4b4ae90 con 0x563cc4c25b50 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.184+0000 7f0c0f400640 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.40:6834/3709744112,v1:172.21.15.40:6835/3709744112] conn(0x563cc4c25b50 0x563cc4c27ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0c0f400640 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.40:6834/3709744112,v1:172.21.15.40:6835/3709744112] conn(0x563cc4c25b50 0x563cc4c27ba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff1d2dd1acebf5c server_cookie=14fc0fda8cf9bbd6 in_seq=0 out_seq=0 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0bfcc00640 10 client.4611 ms_handle_connect on v2:172.21.15.40:6834/3709744112 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0bfcc00640 1 -- 172.21.15.40:0/393714267 <== mds.0 v2:172.21.15.40:6834/3709744112 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x563cc4b4ae90 con 0x563cc4c25b50 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0bfcc00640 10 client.4611 handle_command_reply: tid=0 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0c193ff740 1 client.4611 shutdown 2024-03-08T06:04:18.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0c193ff740 1 -- 172.21.15.40:0/393714267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 msgr2=0x563cc4bde860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:18.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0c193ff740 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bde860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e160 tx=0x7f0bf8009040 comp rx=0 tx=0).stop 2024-03-08T06:04:18.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0c193ff740 1 -- 172.21.15.40:0/393714267 shutdown_connections 2024-03-08T06:04:18.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0c193ff740 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563cc4c1b780 0x563cc4bdeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0c193ff740 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563cc4c06d70 0x563cc4bde860 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0c193ff740 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.40:6834/3709744112,v1:172.21.15.40:6835/3709744112] conn(0x563cc4c25b50 0x563cc4c27ba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0c193ff740 1 --2- 172.21.15.40:0/393714267 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x563cc4c06990 0x563cc4c1e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0c193ff740 1 -- 172.21.15.40:0/393714267 >> 172.21.15.40:0/393714267 conn(0x563cc4bd9e70 msgr2=0x563cc4c04f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:18.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0bfcc00640 0 client.4611 ms_handle_reset on v2:172.21.15.40:6834/3709744112 2024-03-08T06:04:18.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.185+0000 7f0c193ff740 1 -- 172.21.15.40:0/393714267 shutdown_connections 2024-03-08T06:04:18.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c193ff740 1 -- 172.21.15.40:0/393714267 wait complete. 2024-03-08T06:04:18.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c193ff740 20 client.4611 trim_cache size 0 max 16384 2024-03-08T06:04:18.191 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:04:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "c4e20909-a427-4e44-a3fb-93b068b9d1f6", 2024-03-08T06:04:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-03-08T06:04:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4260, 2024-03-08T06:04:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-03-08T06:04:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-03-08T06:04:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-03-08T06:04:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 15.653287157999999, 2024-03-08T06:04:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 8, 2024-03-08T06:04:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 21, 2024-03-08T06:04:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 21, 2024-03-08T06:04:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 23.485535351999999 2024-03-08T06:04:18.192 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c15200640 1 -- 172.21.15.40:0/4276038349 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0bec06b4d0 msgr2=0x7f0bec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c15200640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0bec06b4d0 0x7f0bec06d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c00036000 tx=0x7f0c0003b040 comp rx=0 tx=0).stop 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c15200640 1 -- 172.21.15.40:0/4276038349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c100b44c0 msgr2=0x7f0c1014e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c15200640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c100b44c0 0x7f0c1014e7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800cbb0 tx=0x7f0c0800a430 comp rx=0 tx=0).stop 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c15200640 1 -- 172.21.15.40:0/4276038349 shutdown_connections 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c15200640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f0bec06b4d0 0x7f0bec06d990 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c15200640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c100b44c0 0x7f0c1014e7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c15200640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0c100aee90 0x7f0c101eb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c15200640 1 --2- 172.21.15.40:0/4276038349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10071b10 0x7f0c100adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.186+0000 7f0c15200640 1 -- 172.21.15.40:0/4276038349 >> 172.21.15.40:0/4276038349 conn(0x7f0c100b1590 msgr2=0x7f0c101637e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.187+0000 7f0c15200640 1 -- 172.21.15.40:0/4276038349 shutdown_connections 2024-03-08T06:04:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:18.187+0000 7f0c15200640 1 -- 172.21.15.40:0/4276038349 wait complete. 2024-03-08T06:04:18.204 INFO:teuthology.run_tasks:Running task kclient... 2024-03-08T06:04:18.227 INFO:tasks.kclient:Mounting kernel clients... 2024-03-08T06:04:18.227 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-08T06:04:18.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:18.227 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-03-08T06:04:18.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:18.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-03-08T06:04:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-03-08T06:04:18.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:04:18.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:18.393 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-03-08T06:04:18.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:18.448 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-03-08T06:04:18.552 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-03-08T06:04:18.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:04:18.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:18.559 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-03-08T06:04:18.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:18.618 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-03-08T06:04:18.716 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2024-03-08T06:04:18.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:04:18.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:18.720 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-03-08T06:04:18.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:18.776 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-03-08T06:04:18.878 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2024-03-08T06:04:18.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:04:18.883 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-08T06:04:18.883 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T06:04:18.883 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T06:04:18.883 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:04:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.115+0000 7f4f8e200640 1 Processor -- start 2024-03-08T06:04:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.115+0000 7f4f8e200640 1 -- start start 2024-03-08T06:04:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.117+0000 7f4f8e200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f8815fa00 0x7f4f880b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.117+0000 7f4f8e200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4f880716e0 0x7f4f88071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.117+0000 7f4f8e200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f88162960 con 0x7f4f881624d0 2024-03-08T06:04:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.117+0000 7f4f8e200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f88072090 con 0x7f4f880716e0 2024-03-08T06:04:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.117+0000 7f4f8e200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f88162c40 con 0x7f4f8815fa00 2024-03-08T06:04:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.118+0000 7f4f8d800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4f880716e0 0x7f4f88071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.118+0000 7f4f8ce00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f8815fa00 0x7f4f880b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.118+0000 7f4f8d800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4f880716e0 0x7f4f88071b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:34752/0 (socket says 172.21.15.40:34752) 2024-03-08T06:04:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.118+0000 7f4f8d800640 1 -- 172.21.15.40:0/1224313264 learned_addr learned my addr 172.21.15.40:0/1224313264 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.119+0000 7f4f87e00640 1 -- 172.21.15.40:0/1224313264 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409297363 0 0) 0x7f4f88162960 con 0x7f4f881624d0 2024-03-08T06:04:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.119+0000 7f4f87e00640 1 -- 172.21.15.40:0/1224313264 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f6c003520 con 0x7f4f881624d0 2024-03-08T06:04:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.119+0000 7f4f8d800640 1 -- 172.21.15.40:0/1224313264 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f8815fa00 msgr2=0x7f4f880b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.119+0000 7f4f8d800640 1 --2- 172.21.15.40:0/1224313264 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f8815fa00 0x7f4f880b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.119+0000 7f4f8d800640 1 -- 172.21.15.40:0/1224313264 >> v1:172.21.15.53:6789/0 conn(0x7f4f881624d0 legacy=0x7f4f880b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.119+0000 7f4f8d800640 1 -- 172.21.15.40:0/1224313264 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f88162f20 con 0x7f4f880716e0 2024-03-08T06:04:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.119+0000 7f4f8ce00640 1 --2- 172.21.15.40:0/1224313264 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f8815fa00 0x7f4f880b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.120+0000 7f4f8d800640 1 --2- 172.21.15.40:0/1224313264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f880716e0 0x7f4f88071b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009e50 tx=0x7f4f8002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e5776a0a6fd1eba server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.121+0000 7f4f87e00640 1 -- 172.21.15.40:0/1224313264 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8002ca80 con 0x7f4f880716e0 2024-03-08T06:04:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.121+0000 7f4f87e00640 1 -- 172.21.15.40:0/1224313264 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8002cc20 con 0x7f4f880716e0 2024-03-08T06:04:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.121+0000 7f4f87e00640 1 -- 172.21.15.40:0/1224313264 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f800343f0 con 0x7f4f880716e0 2024-03-08T06:04:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.121+0000 7f4f8e200640 1 -- 172.21.15.40:0/1224313264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f880716e0 msgr2=0x7f4f88071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.121+0000 7f4f8e200640 1 --2- 172.21.15.40:0/1224313264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f880716e0 0x7f4f88071b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009e50 tx=0x7f4f8002bee0 comp rx=0 tx=0).stop 2024-03-08T06:04:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.121+0000 7f4f8e200640 1 -- 172.21.15.40:0/1224313264 shutdown_connections 2024-03-08T06:04:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.121+0000 7f4f8e200640 1 --2- 172.21.15.40:0/1224313264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f880716e0 0x7f4f88071b50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.121+0000 7f4f8e200640 1 --2- 172.21.15.40:0/1224313264 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f8815fa00 0x7f4f880b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.121+0000 7f4f8e200640 1 -- 172.21.15.40:0/1224313264 >> 172.21.15.40:0/1224313264 conn(0x7f4f880b1850 msgr2=0x7f4f880b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.121+0000 7f4f8e200640 1 -- 172.21.15.40:0/1224313264 shutdown_connections 2024-03-08T06:04:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.121+0000 7f4f8e200640 1 -- 172.21.15.40:0/1224313264 wait complete. 2024-03-08T06:04:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.123+0000 7f4f8e200640 1 Processor -- start 2024-03-08T06:04:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.123+0000 7f4f8e200640 1 -- start start 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.124+0000 7f4f8e200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f880716e0 0x7f4f88145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.124+0000 7f4f8e200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f8815fa00 0x7f4f8806ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.124+0000 7f4f8e200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f8806b190 0x7f4f881f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.124+0000 7f4f8e200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f880b4d60 con 0x7f4f880716e0 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.124+0000 7f4f8e200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8815fde0 con 0x7f4f8815fa00 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.124+0000 7f4f8e200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f880b2eb0 con 0x7f4f8806b190 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.124+0000 7f4f8ce00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f880716e0 0x7f4f88145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.124+0000 7f4f8ce00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f880716e0 0x7f4f88145900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56162/0 (socket says 172.21.15.40:56162) 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.125+0000 7f4f8ce00640 1 -- 172.21.15.40:0/3723518171 learned_addr learned my addr 172.21.15.40:0/3723518171 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.125+0000 7f4f7fe00640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f8815fa00 0x7f4f8806ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.125+0000 7f4f8d800640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f8806b190 0x7f4f881f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.125+0000 7f4f8ce00640 1 -- 172.21.15.40:0/3723518171 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f8806b190 msgr2=0x7f4f881f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.125+0000 7f4f8ce00640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f8806b190 0x7f4f881f7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.125+0000 7f4f8ce00640 1 -- 172.21.15.40:0/3723518171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f8815fa00 msgr2=0x7f4f8806ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.125+0000 7f4f8ce00640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f8815fa00 0x7f4f8806ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.125+0000 7f4f8ce00640 1 -- 172.21.15.40:0/3723518171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f881598a0 con 0x7f4f880716e0 2024-03-08T06:04:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.125+0000 7f4f8ce00640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f880716e0 0x7f4f88145900 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800e960 tx=0x7f4f7800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.125+0000 7f4f85600640 1 -- 172.21.15.40:0/3723518171 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78017070 con 0x7f4f880716e0 2024-03-08T06:04:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.126+0000 7f4f85600640 1 -- 172.21.15.40:0/3723518171 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7800ca00 con 0x7f4f880716e0 2024-03-08T06:04:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.126+0000 7f4f85600640 1 -- 172.21.15.40:0/3723518171 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800cdd0 con 0x7f4f880716e0 2024-03-08T06:04:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.126+0000 7f4f8e200640 1 -- 172.21.15.40:0/3723518171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f88159b80 con 0x7f4f880716e0 2024-03-08T06:04:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.126+0000 7f4f8e200640 1 -- 172.21.15.40:0/3723518171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f88145e40 con 0x7f4f880716e0 2024-03-08T06:04:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.127+0000 7f4f8e200640 1 -- 172.21.15.40:0/3723518171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8806b5a0 con 0x7f4f880716e0 2024-03-08T06:04:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.128+0000 7f4f85600640 1 -- 172.21.15.40:0/3723518171 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f7801a070 con 0x7f4f880716e0 2024-03-08T06:04:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.129+0000 7f4f85600640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4f6406b4d0 0x7f4f6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.129+0000 7f4f85600640 1 -- 172.21.15.40:0/3723518171 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f7808a510 con 0x7f4f880716e0 2024-03-08T06:04:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.130+0000 7f4f7fe00640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4f6406b4d0 0x7f4f6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.130+0000 7f4f7fe00640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4f6406b4d0 0x7f4f6406d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4f880b3d10 tx=0x7f4f70009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:19.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.134+0000 7f4f85600640 1 -- 172.21.15.40:0/3723518171 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f8806b5a0 con 0x7f4f880716e0 2024-03-08T06:04:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.371+0000 7f4f8e200640 1 -- 172.21.15.40:0/3723518171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4f88075100 con 0x7f4f880716e0 2024-03-08T06:04:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.372+0000 7f4f85600640 1 -- 172.21.15.40:0/3723518171 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4f780592f0 con 0x7f4f880716e0 2024-03-08T06:04:19.372 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T06:04:19.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.374+0000 7f4f8e200640 1 -- 172.21.15.40:0/3723518171 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4f6406b4d0 msgr2=0x7f4f6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.374+0000 7f4f8e200640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4f6406b4d0 0x7f4f6406d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4f880b3d10 tx=0x7f4f70009040 comp rx=0 tx=0).stop 2024-03-08T06:04:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.374+0000 7f4f8e200640 1 -- 172.21.15.40:0/3723518171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f880716e0 msgr2=0x7f4f88145900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.374+0000 7f4f8e200640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f880716e0 0x7f4f88145900 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800e960 tx=0x7f4f7800ee20 comp rx=0 tx=0).stop 2024-03-08T06:04:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.374+0000 7f4f8e200640 1 -- 172.21.15.40:0/3723518171 shutdown_connections 2024-03-08T06:04:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.374+0000 7f4f8e200640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4f6406b4d0 0x7f4f6406d990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.374+0000 7f4f8e200640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f8806b190 0x7f4f881f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.374+0000 7f4f8e200640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f8815fa00 0x7f4f8806ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.374+0000 7f4f8e200640 1 --2- 172.21.15.40:0/3723518171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f880716e0 0x7f4f88145900 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.374+0000 7f4f8e200640 1 -- 172.21.15.40:0/3723518171 >> 172.21.15.40:0/3723518171 conn(0x7f4f880b1850 msgr2=0x7f4f880b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.374+0000 7f4f8e200640 1 -- 172.21.15.40:0/3723518171 shutdown_connections 2024-03-08T06:04:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:19.375+0000 7f4f8e200640 1 -- 172.21.15.40:0/3723518171 wait complete. 2024-03-08T06:04:19.392 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T06:04:19.392 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:04:19.392 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-03-08T06:04:19.392 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-08T06:04:19.393 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T06:04:19.393 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T06:04:19.393 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T06:04:19.393 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:04:19.393 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-08T06:04:19.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:19.393 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-03-08T06:04:19.442 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:04:19.442 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:04:19.442 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:04:19.442 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:19.442 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-03-08T06:04:19.442 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42506sec preferred_lft 42506sec 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:19.443 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-03-08T06:04:19.444 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-08T06:04:19.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:19.445 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-03-08T06:04:19.445 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-03-08T06:04:19.445 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2024-03-08T06:04:19.445 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2024-03-08T06:04:19.445 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2024-03-08T06:04:19.445 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-08T06:04:19.445 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-03-08T06:04:19.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:19.884 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:04:19.970 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-03-08T06:04:19.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:19.973 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-03-08T06:04:20.029 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-03-08T06:04:20.029 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:04:20.029 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-08T06:04:20.029 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-08T06:04:20.029 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:04:20.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:20.030 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-03-08T06:04:20.030 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-03-08T06:04:20.030 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-08T06:04:20.030 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-08T06:04:20.030 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-08T06:04:20.030 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-03-08T06:04:20.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:20.778 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-03-08T06:04:20.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:20.837 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-03-08T06:04:20.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:20.896 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-03-08T06:04:20.896 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-03-08T06:04:20.896 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T06:04:20.896 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-08T06:04:20.897 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-03-08T06:04:21.048 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-08T06:04:21.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:21.048 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-03-08T06:04:21.048 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-03-08T06:04:21.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-08T06:04:21.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-08T06:04:21.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-08T06:04:21.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-08T06:04:21.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-08T06:04:21.049 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-03-08T06:04:21.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:21.328 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-03-08T06:04:21.328 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-03-08T06:04:21.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-03-08T06:04:21.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2024-03-08T06:04:21.328 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-03-08T06:04:21.497 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:04:21.497 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-08T06:04:21.548 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-08T06:04:21.548 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:04:21.548 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-08T06:04:21.599 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-08T06:04:21.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:21.599 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-08T06:04:21.744 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-08T06:04:21.744 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-08T06:04:21.744 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-03-08T06:04:22.370 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-08T06:04:22.371 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-08T06:04:22.371 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-03-08T06:04:22.371 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:04:22.371 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:04:22.406 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-08T06:04:22.406 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-08T06:04:22.406 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-03-08T06:04:22.407 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:04:22.407 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:04:22.407 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790" 2024-03-08T06:04:22.407 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790:/ 2024-03-08T06:04:22.407 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4e20909-a427-4e44-a3fb-93b068b9d1f6" will pass to kernel 2024-03-08T06:04:22.413 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-08T06:04:22.413 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:04:22.413 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:04:22.413 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:04:22.413 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:04:22.419 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:04:22.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:22.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-08T06:04:22.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:22.520 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:22.589 INFO:teuthology.orchestra.run.smithi040.stdout:client4409 2024-03-08T06:04:22.589 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:04:22.589 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:22.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:22.644 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:22.713 INFO:teuthology.orchestra.run.smithi040.stdout:client4409 2024-03-08T06:04:22.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:22.713 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:22.789 INFO:teuthology.orchestra.run.smithi040.stdout:client4409 2024-03-08T06:04:22.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:22.789 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c4e20909-a427-4e44-a3fb-93b068b9d1f6.client4409/status') 2024-03-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.4409 (3)172.21.15.40:0/1450261440 2024-03-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2024-03-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2024-03-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2024-03-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:70 bytes copied, 7.1505e-05 s, 979 kB/s 2024-03-08T06:04:22.902 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-03-08T06:04:22.902 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T06:04:22.902 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-08T06:04:22.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:04:23.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.127+0000 7f2014c00640 1 Processor -- start 2024-03-08T06:04:23.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.127+0000 7f2014c00640 1 -- start start 2024-03-08T06:04:23.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.128+0000 7f2014c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f201015df20 0x7f20100ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:23.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.128+0000 7f2014c00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f20100d11b0 0x7f2010162400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.128+0000 7f2014c00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f201015e300 con 0x7f20100b6bd0 2024-03-08T06:04:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.128+0000 7f2014c00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20100d3990 con 0x7f20100d11b0 2024-03-08T06:04:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.128+0000 7f2014c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2010162940 con 0x7f201015df20 2024-03-08T06:04:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.129+0000 7f200fe00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f20100d11b0 0x7f2010162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.129+0000 7f200f400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f201015df20 0x7f20100ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.129+0000 7f200f400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f201015df20 0x7f20100ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:40658/0 (socket says 172.21.15.40:40658) 2024-03-08T06:04:23.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.129+0000 7f200fe00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f20100d11b0 0x7f2010162400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:34772/0 (socket says 172.21.15.40:34772) 2024-03-08T06:04:23.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.129+0000 7f200f400640 1 -- 172.21.15.40:0/1714460489 learned_addr learned my addr 172.21.15.40:0/1714460489 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:23.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.130+0000 7f200f400640 1 -- 172.21.15.40:0/1714460489 >> v2:172.21.15.53:3300/0 conn(0x7f20100d11b0 msgr2=0x7f2010162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:23.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.130+0000 7f200f400640 1 --2- 172.21.15.40:0/1714460489 >> v2:172.21.15.53:3300/0 conn(0x7f20100d11b0 0x7f2010162400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:23.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.130+0000 7f200f400640 1 -- 172.21.15.40:0/1714460489 >> v1:172.21.15.53:6789/0 conn(0x7f20100b6bd0 legacy=0x7f20100d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:23.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.130+0000 7f200f400640 1 -- 172.21.15.40:0/1714460489 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20100cf030 con 0x7f201015df20 2024-03-08T06:04:23.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.130+0000 7f200fe00640 1 --2- 172.21.15.40:0/1714460489 >> v2:172.21.15.53:3300/0 conn(0x7f20100d11b0 0x7f2010162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:04:23.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.132+0000 7f200f400640 1 --2- 172.21.15.40:0/1714460489 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f201015df20 0x7f20100ce9e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2000009e50 tx=0x7f200002bfe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccacc3cb1224dc1d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:23.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.132+0000 7f200ea00640 1 -- 172.21.15.40:0/1714460489 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200002cb80 con 0x7f201015df20 2024-03-08T06:04:23.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.132+0000 7f200ea00640 1 -- 172.21.15.40:0/1714460489 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200002cd20 con 0x7f201015df20 2024-03-08T06:04:23.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.132+0000 7f200ea00640 1 -- 172.21.15.40:0/1714460489 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000034610 con 0x7f201015df20 2024-03-08T06:04:23.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.133+0000 7f2014c00640 1 -- 172.21.15.40:0/1714460489 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f201015df20 msgr2=0x7f20100ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:23.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.133+0000 7f2014c00640 1 --2- 172.21.15.40:0/1714460489 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f201015df20 0x7f20100ce9e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2000009e50 tx=0x7f200002bfe0 comp rx=0 tx=0).stop 2024-03-08T06:04:23.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.133+0000 7f2014c00640 1 -- 172.21.15.40:0/1714460489 shutdown_connections 2024-03-08T06:04:23.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.133+0000 7f2014c00640 1 --2- 172.21.15.40:0/1714460489 >> v2:172.21.15.53:3300/0 conn(0x7f20100d11b0 0x7f2010162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:23.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.133+0000 7f2014c00640 1 --2- 172.21.15.40:0/1714460489 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f201015df20 0x7f20100ce9e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:23.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.133+0000 7f2014c00640 1 -- 172.21.15.40:0/1714460489 >> 172.21.15.40:0/1714460489 conn(0x7f201006c740 msgr2=0x7f201006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:23.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.133+0000 7f2014c00640 1 -- 172.21.15.40:0/1714460489 shutdown_connections 2024-03-08T06:04:23.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.133+0000 7f2014c00640 1 -- 172.21.15.40:0/1714460489 wait complete. 2024-03-08T06:04:23.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.135+0000 7f2014c00640 1 Processor -- start 2024-03-08T06:04:23.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.135+0000 7f2014c00640 1 -- start start 2024-03-08T06:04:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.136+0000 7f2014c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20100b6bd0 0x7f201015c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.136+0000 7f2014c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20100d11b0 0x7f2010160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.136+0000 7f2014c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201015df20 0x7f2010154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.136+0000 7f2014c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20100d2180 con 0x7f20100b6bd0 2024-03-08T06:04:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.136+0000 7f2014c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20100d1ea0 con 0x7f201015df20 2024-03-08T06:04:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.136+0000 7f2014c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20100d2460 con 0x7f20100d11b0 2024-03-08T06:04:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f200f400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20100b6bd0 0x7f201015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f200f400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20100b6bd0 0x7f201015c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56174/0 (socket says 172.21.15.40:56174) 2024-03-08T06:04:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f200f400640 1 -- 172.21.15.40:0/668537993 learned_addr learned my addr 172.21.15.40:0/668537993 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f2006a00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20100d11b0 0x7f2010160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f200fe00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201015df20 0x7f2010154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f200f400640 1 -- 172.21.15.40:0/668537993 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20100d11b0 msgr2=0x7f2010160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f200f400640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20100d11b0 0x7f2010160cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f200f400640 1 -- 172.21.15.40:0/668537993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201015df20 msgr2=0x7f2010154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f200f400640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201015df20 0x7f2010154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f200f400640 1 -- 172.21.15.40:0/668537993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20100c8430 con 0x7f20100b6bd0 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f2006a00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20100d11b0 0x7f2010160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.137+0000 7f200fe00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201015df20 0x7f2010154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.138+0000 7f200f400640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20100b6bd0 0x7f201015c680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2000009e50 tx=0x7f2000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.138+0000 7f2007e00640 1 -- 172.21.15.40:0/668537993 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000043020 con 0x7f20100b6bd0 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.138+0000 7f2007e00640 1 -- 172.21.15.40:0/668537993 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200003e450 con 0x7f20100b6bd0 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.138+0000 7f2007e00640 1 -- 172.21.15.40:0/668537993 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200003e840 con 0x7f20100b6bd0 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.138+0000 7f2014c00640 1 -- 172.21.15.40:0/668537993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20100c89f0 con 0x7f20100b6bd0 2024-03-08T06:04:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.138+0000 7f2014c00640 1 -- 172.21.15.40:0/668537993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20100c8fb0 con 0x7f20100b6bd0 2024-03-08T06:04:23.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.139+0000 7f2014c00640 1 -- 172.21.15.40:0/668537993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2010075100 con 0x7f20100b6bd0 2024-03-08T06:04:23.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.140+0000 7f2007e00640 1 -- 172.21.15.40:0/668537993 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2000046020 con 0x7f20100b6bd0 2024-03-08T06:04:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.141+0000 7f2007e00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1fec06b4d0 0x7f1fec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.142+0000 7f2007e00640 1 -- 172.21.15.40:0/668537993 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20000acaa0 con 0x7f20100b6bd0 2024-03-08T06:04:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.142+0000 7f2006a00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1fec06b4d0 0x7f1fec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.142+0000 7f2006a00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1fec06b4d0 0x7f1fec06d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f20100cdae0 tx=0x7f1ff8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.147+0000 7f2007e00640 1 -- 172.21.15.40:0/668537993 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f200004d030 con 0x7f20100b6bd0 2024-03-08T06:04:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.387+0000 7f2014c00640 1 -- 172.21.15.40:0/668537993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f20100d16e0 con 0x7f20100b6bd0 2024-03-08T06:04:23.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.388+0000 7f2007e00640 1 -- 172.21.15.40:0/668537993 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f200007b940 con 0x7f20100b6bd0 2024-03-08T06:04:23.389 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T06:04:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.390+0000 7f2014c00640 1 -- 172.21.15.40:0/668537993 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1fec06b4d0 msgr2=0x7f1fec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.390+0000 7f2014c00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1fec06b4d0 0x7f1fec06d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f20100cdae0 tx=0x7f1ff8009040 comp rx=0 tx=0).stop 2024-03-08T06:04:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.391+0000 7f2014c00640 1 -- 172.21.15.40:0/668537993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20100b6bd0 msgr2=0x7f201015c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.391+0000 7f2014c00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20100b6bd0 0x7f201015c680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2000009e50 tx=0x7f2000004a40 comp rx=0 tx=0).stop 2024-03-08T06:04:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.391+0000 7f2014c00640 1 -- 172.21.15.40:0/668537993 shutdown_connections 2024-03-08T06:04:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.391+0000 7f2014c00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1fec06b4d0 0x7f1fec06d990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.391+0000 7f2014c00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201015df20 0x7f2010154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.391+0000 7f2014c00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20100d11b0 0x7f2010160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.391+0000 7f2014c00640 1 --2- 172.21.15.40:0/668537993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20100b6bd0 0x7f201015c680 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.391+0000 7f2014c00640 1 -- 172.21.15.40:0/668537993 >> 172.21.15.40:0/668537993 conn(0x7f201006c740 msgr2=0x7f20100d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.391+0000 7f2014c00640 1 -- 172.21.15.40:0/668537993 shutdown_connections 2024-03-08T06:04:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:23.391+0000 7f2014c00640 1 -- 172.21.15.40:0/668537993 wait complete. 2024-03-08T06:04:23.410 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T06:04:23.410 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:04:23.410 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-03-08T06:04:23.410 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-08T06:04:23.410 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-08T06:04:23.410 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T06:04:23.410 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T06:04:23.410 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:04:23.410 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-08T06:04:23.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:23.411 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-03-08T06:04:23.463 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:04:23.463 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42502sec preferred_lft 42502sec 2024-03-08T06:04:23.464 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:59:73:2d:1a:29 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::18e3:96ff:fe16:9a48/64 scope link 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:59:73:2d:1a:29 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f859:73ff:fe2d:1a29/64 scope link 2024-03-08T06:04:23.465 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:23.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:23.466 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:04:23.553 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-03-08T06:04:23.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:23.558 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-03-08T06:04:23.615 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-03-08T06:04:23.615 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:04:23.615 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-08T06:04:23.616 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-08T06:04:23.616 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:04:23.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:23.617 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-03-08T06:04:23.617 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-03-08T06:04:23.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-08T06:04:23.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-08T06:04:23.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-08T06:04:23.617 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-03-08T06:04:23.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:23.812 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-03-08T06:04:23.864 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-08T06:04:23.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:23.865 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-03-08T06:04:23.918 INFO:teuthology.orchestra.run.smithi040.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-08T06:04:23.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:23.919 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-03-08T06:04:23.919 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-03-08T06:04:23.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-08T06:04:23.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-08T06:04:23.920 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-03-08T06:04:24.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:24.068 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-08T06:04:24.163 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:04:24.163 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:04:24.164 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:04:24.164 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:24.164 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-03-08T06:04:24.164 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:24.164 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:04:24.164 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 8a:56:05:0c:9d:54 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T06:04:24.164 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T06:04:24.164 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:24.164 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::8856:5ff:fe0c:9d54/64 scope link 2024-03-08T06:04:24.164 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:24.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:24.169 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-08T06:04:24.266 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:04:24.266 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:04:24.266 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:04:24.266 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:24.266 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-03-08T06:04:24.266 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:24.266 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:04:24.266 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 8a:56:05:0c:9d:54 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T06:04:24.266 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T06:04:24.266 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:24.267 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::8856:5ff:fe0c:9d54/64 scope link 2024-03-08T06:04:24.267 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:24.271 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-08T06:04:24.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:24.271 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-03-08T06:04:24.271 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-03-08T06:04:24.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-08T06:04:24.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-08T06:04:24.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-08T06:04:24.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-08T06:04:24.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-08T06:04:24.272 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-03-08T06:04:24.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:24.567 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-03-08T06:04:24.568 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-03-08T06:04:24.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.1 up 2024-03-08T06:04:24.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.1 master ceph-brx 2024-03-08T06:04:24.568 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-03-08T06:04:24.744 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:04:24.744 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-08T06:04:24.792 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-08T06:04:24.792 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:04:24.793 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-08T06:04:24.842 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-08T06:04:24.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:24.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-08T06:04:24.990 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-08T06:04:24.990 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-08T06:04:24.991 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-03-08T06:04:25.013 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-08T06:04:25.013 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-08T06:04:25.014 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-03-08T06:04:25.014 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:04:25.014 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:04:25.040 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-08T06:04:25.040 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-08T06:04:25.041 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-03-08T06:04:25.041 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:04:25.041 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:04:25.041 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790" 2024-03-08T06:04:25.041 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790:/ 2024-03-08T06:04:25.041 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=c4e20909-a427-4e44-a3fb-93b068b9d1f6" will pass to kernel 2024-03-08T06:04:25.043 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-08T06:04:25.043 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:04:25.043 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:04:25.043 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:04:25.043 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:04:25.047 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:04:25.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:25.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-08T06:04:25.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:25.146 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:25.207 INFO:teuthology.orchestra.run.smithi040.stdout:client4620 2024-03-08T06:04:25.208 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:04:25.208 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:25.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:25.264 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:25.322 INFO:teuthology.orchestra.run.smithi040.stdout:client4620 2024-03-08T06:04:25.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:25.323 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:25.384 INFO:teuthology.orchestra.run.smithi040.stdout:client4620 2024-03-08T06:04:25.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:25.384 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c4e20909-a427-4e44-a3fb-93b068b9d1f6.client4620/status') 2024-03-08T06:04:25.478 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.4620 (3)192.168.0.2:0/2309351825 2024-03-08T06:04:25.478 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2024-03-08T06:04:25.478 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2024-03-08T06:04:25.479 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2024-03-08T06:04:25.479 INFO:teuthology.orchestra.run.smithi040.stderr:69 bytes copied, 6.7108e-05 s, 1.0 MB/s 2024-03-08T06:04:25.483 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-08T06:04:25.483 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T06:04:25.483 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-08T06:04:25.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:04:25.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.706+0000 7f826be00640 1 Processor -- start 2024-03-08T06:04:25.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.707+0000 7f826be00640 1 -- start start 2024-03-08T06:04:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.708+0000 7f826be00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82640d3630 0x7f82640d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.708+0000 7f826be00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82640d2b90 con 0x7f82641614f0 2024-03-08T06:04:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.708+0000 7f826be00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82640b9820 con 0x7f82640cf020 2024-03-08T06:04:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.708+0000 7f826be00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82640b6d50 con 0x7f82640d3630 2024-03-08T06:04:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.708+0000 7f8269400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f82641614f0 0x7f82640ce910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38142/0 (socket says 172.21.15.40:38142) 2024-03-08T06:04:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.708+0000 7f8269400640 1 -- 172.21.15.40:0/716047826 learned_addr learned my addr 172.21.15.40:0/716047826 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8269e00640 1 --2- 172.21.15.40:0/716047826 >> v2:172.21.15.53:3300/0 conn(0x7f82640d3630 0x7f82640d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177241674 0 0) 0x7f82640d2b90 con 0x7f82641614f0 2024-03-08T06:04:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f824c003520 con 0x7f82641614f0 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 275729822 0 0) 0x7f824c003520 con 0x7f82641614f0 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 >> v2:172.21.15.53:3300/0 conn(0x7f82640d3630 msgr2=0x7f82640d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8263e00640 1 --2- 172.21.15.40:0/716047826 >> v2:172.21.15.53:3300/0 conn(0x7f82640d3630 0x7f82640d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 >> v1:172.21.15.53:6789/0 conn(0x7f82640cf020 legacy=0x7f82640d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640b9b00 con 0x7f82641614f0 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f8258001670 con 0x7f82641614f0 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 >> v1:172.21.15.40:6789/0 conn(0x7f82641614f0 legacy=0x7f82640ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8263e00640 1 --2- 172.21.15.40:0/716047826 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f824c003a00 0x7f824c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.709+0000 7f8263e00640 1 --2- 172.21.15.40:0/716047826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f824c004800 0x7f824c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8263e00640 1 --2- 172.21.15.40:0/716047826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 0x7f824c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824c00c090 con 0x7f824c0051c0 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824c00c240 con 0x7f824c004800 2024-03-08T06:04:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824c00c440 con 0x7f824c003a00 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8269e00640 1 --2- 172.21.15.40:0/716047826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 0x7f824c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8269400640 1 --2- 172.21.15.40:0/716047826 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f824c003a00 0x7f824c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8268a00640 1 --2- 172.21.15.40:0/716047826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f824c004800 0x7f824c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8269e00640 1 -- 172.21.15.40:0/716047826 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f824c003a00 msgr2=0x7f824c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8269e00640 1 --2- 172.21.15.40:0/716047826 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f824c003a00 0x7f824c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8269e00640 1 -- 172.21.15.40:0/716047826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f824c004800 msgr2=0x7f824c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8269e00640 1 --2- 172.21.15.40:0/716047826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f824c004800 0x7f824c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.710+0000 7f8269e00640 1 -- 172.21.15.40:0/716047826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824c003520 con 0x7f824c0051c0 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.711+0000 7f8269e00640 1 -- 172.21.15.40:0/716047826 reap_dead start 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.712+0000 7f8269e00640 1 --2- 172.21.15.40:0/716047826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 0x7f824c00b9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f825c009fc0 tx=0x7f824c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b325c83e1f8fda server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.712+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c009c10 con 0x7f824c0051c0 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.712+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c02cca0 con 0x7f824c0051c0 2024-03-08T06:04:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.712+0000 7f8263e00640 1 -- 172.21.15.40:0/716047826 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c003bd0 con 0x7f824c0051c0 2024-03-08T06:04:25.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.712+0000 7f826be00640 1 -- 172.21.15.40:0/716047826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 msgr2=0x7f824c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:25.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.712+0000 7f826be00640 1 --2- 172.21.15.40:0/716047826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 0x7f824c00b9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f825c009fc0 tx=0x7f824c005830 comp rx=0 tx=0).stop 2024-03-08T06:04:25.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.712+0000 7f826be00640 1 -- 172.21.15.40:0/716047826 shutdown_connections 2024-03-08T06:04:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.712+0000 7f826be00640 1 --2- 172.21.15.40:0/716047826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 0x7f824c00b9c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.712+0000 7f826be00640 1 -- 172.21.15.40:0/716047826 >> 172.21.15.40:0/716047826 conn(0x7f826406c8f0 msgr2=0x7f826406cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.713+0000 7f826be00640 1 -- 172.21.15.40:0/716047826 shutdown_connections 2024-03-08T06:04:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.713+0000 7f826be00640 1 -- 172.21.15.40:0/716047826 wait complete. 2024-03-08T06:04:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.714+0000 7f826be00640 1 Processor -- start 2024-03-08T06:04:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.715+0000 7f826be00640 1 -- start start 2024-03-08T06:04:25.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.715+0000 7f826be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 0x7f82640cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:25.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.715+0000 7f826be00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826414d520 0x7f826414cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:25.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f826be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8264149c50 0x7f826414a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:25.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f826be00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640b95d0 con 0x7f824c0051c0 2024-03-08T06:04:25.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f826be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640d3370 con 0x7f8264149c50 2024-03-08T06:04:25.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f826be00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82640d2ee0 con 0x7f826414d520 2024-03-08T06:04:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f8269400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 0x7f82640cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f8269400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 0x7f82640cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56192/0 (socket says 172.21.15.40:56192) 2024-03-08T06:04:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f8269400640 1 -- 172.21.15.40:0/3255966978 learned_addr learned my addr 172.21.15.40:0/3255966978 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f8268a00640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826414d520 0x7f826414cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f8269400640 1 -- 172.21.15.40:0/3255966978 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826414d520 msgr2=0x7f826414cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f8269400640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826414d520 0x7f826414cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f8269400640 1 -- 172.21.15.40:0/3255966978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8264149c50 msgr2=0x7f826414a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f8269400640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8264149c50 0x7f826414a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f8269400640 1 -- 172.21.15.40:0/3255966978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826414a880 con 0x7f824c0051c0 2024-03-08T06:04:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.716+0000 7f8269400640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 0x7f82640cc700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f825800e730 tx=0x7f825800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:25.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.717+0000 7f8261600640 1 -- 172.21.15.40:0/3255966978 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825800c6c0 con 0x7f824c0051c0 2024-03-08T06:04:25.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.717+0000 7f8261600640 1 -- 172.21.15.40:0/3255966978 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8258018070 con 0x7f824c0051c0 2024-03-08T06:04:25.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.717+0000 7f8261600640 1 -- 172.21.15.40:0/3255966978 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825800caa0 con 0x7f824c0051c0 2024-03-08T06:04:25.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.717+0000 7f826be00640 1 -- 172.21.15.40:0/3255966978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82640ccf50 con 0x7f824c0051c0 2024-03-08T06:04:25.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.717+0000 7f826be00640 1 -- 172.21.15.40:0/3255966978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82640cd510 con 0x7f824c0051c0 2024-03-08T06:04:25.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.718+0000 7f826be00640 1 -- 172.21.15.40:0/3255966978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8264067cb0 con 0x7f824c0051c0 2024-03-08T06:04:25.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.719+0000 7f8261600640 1 -- 172.21.15.40:0/3255966978 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8258013420 con 0x7f824c0051c0 2024-03-08T06:04:25.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.721+0000 7f8261600640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824806b500 0x7f824806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:25.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.721+0000 7f8261600640 1 -- 172.21.15.40:0/3255966978 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8258089f90 con 0x7f824c0051c0 2024-03-08T06:04:25.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.721+0000 7f8268a00640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824806b500 0x7f824806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.722+0000 7f8268a00640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824806b500 0x7f824806d9c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8254000c00 tx=0x7f8254009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:25.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.726+0000 7f8261600640 1 -- 172.21.15.40:0/3255966978 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8258058e00 con 0x7f824c0051c0 2024-03-08T06:04:25.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.963+0000 7f826be00640 1 -- 172.21.15.40:0/3255966978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f82640cee00 con 0x7f824c0051c0 2024-03-08T06:04:25.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.964+0000 7f8261600640 1 -- 172.21.15.40:0/3255966978 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8258058700 con 0x7f824c0051c0 2024-03-08T06:04:25.965 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T06:04:25.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.966+0000 7f826be00640 1 -- 172.21.15.40:0/3255966978 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824806b500 msgr2=0x7f824806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:25.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.966+0000 7f826be00640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824806b500 0x7f824806d9c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8254000c00 tx=0x7f8254009040 comp rx=0 tx=0).stop 2024-03-08T06:04:25.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.966+0000 7f826be00640 1 -- 172.21.15.40:0/3255966978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 msgr2=0x7f82640cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:25.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.966+0000 7f826be00640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 0x7f82640cc700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f825800e730 tx=0x7f825800ebf0 comp rx=0 tx=0).stop 2024-03-08T06:04:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.966+0000 7f826be00640 1 -- 172.21.15.40:0/3255966978 shutdown_connections 2024-03-08T06:04:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.967+0000 7f826be00640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824806b500 0x7f824806d9c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.967+0000 7f826be00640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8264149c50 0x7f826414a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.967+0000 7f826be00640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826414d520 0x7f826414cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.967+0000 7f826be00640 1 --2- 172.21.15.40:0/3255966978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c0051c0 0x7f82640cc700 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.967+0000 7f826be00640 1 -- 172.21.15.40:0/3255966978 >> 172.21.15.40:0/3255966978 conn(0x7f826406c8f0 msgr2=0x7f82640c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:25.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.967+0000 7f826be00640 1 -- 172.21.15.40:0/3255966978 shutdown_connections 2024-03-08T06:04:25.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:25.967+0000 7f826be00640 1 -- 172.21.15.40:0/3255966978 wait complete. 2024-03-08T06:04:25.985 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T06:04:25.985 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:04:25.985 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-03-08T06:04:25.985 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-08T06:04:25.986 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-08T06:04:25.986 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T06:04:25.986 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T06:04:25.986 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:04:25.986 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-08T06:04:25.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:25.986 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-03-08T06:04:26.047 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:04:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:04:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:04:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:26.047 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42548sec preferred_lft 42548sec 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:26.048 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-03-08T06:04:26.049 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-08T06:04:26.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:26.049 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2024-03-08T06:04:26.050 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-03-08T06:04:26.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add name ceph-brx type bridge 2024-03-08T06:04:26.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr flush dev ceph-brx 2024-03-08T06:04:26.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set ceph-brx up 2024-03-08T06:04:26.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-08T06:04:26.050 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-03-08T06:04:26.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:26.489 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:04:26.574 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-03-08T06:04:26.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:26.578 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-03-08T06:04:26.624 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-03-08T06:04:26.625 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:04:26.625 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-08T06:04:26.625 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-08T06:04:26.625 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:04:26.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:26.626 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2024-03-08T06:04:26.626 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-03-08T06:04:26.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-08T06:04:26.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-08T06:04:26.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-08T06:04:26.626 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-03-08T06:04:27.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:27.327 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-03-08T06:04:27.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:27.382 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2024-03-08T06:04:27.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:27.432 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2024-03-08T06:04:27.432 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-03-08T06:04:27.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T06:04:27.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-08T06:04:27.432 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-03-08T06:04:27.560 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-08T06:04:27.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:27.560 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2024-03-08T06:04:27.560 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-03-08T06:04:27.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-08T06:04:27.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-08T06:04:27.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-08T06:04:27.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-08T06:04:27.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-08T06:04:27.561 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-03-08T06:04:27.683 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:04:27.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:04:27.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:27.844 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2024-03-08T06:04:27.844 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-03-08T06:04:27.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-03-08T06:04:27.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.0 master ceph-brx 2024-03-08T06:04:27.844 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-03-08T06:04:28.047 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:04:28.047 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-08T06:04:28.089 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-08T06:04:28.089 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:04:28.089 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-08T06:04:28.176 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-08T06:04:28.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:28.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-08T06:04:28.312 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-08T06:04:28.312 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-08T06:04:28.312 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-03-08T06:04:28.976 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-08T06:04:28.976 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-08T06:04:28.976 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-03-08T06:04:28.976 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:04:28.976 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:04:29.010 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-08T06:04:29.010 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-08T06:04:29.010 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-03-08T06:04:29.010 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:04:29.010 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:04:29.010 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790" 2024-03-08T06:04:29.010 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790:/ 2024-03-08T06:04:29.010 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c4e20909-a427-4e44-a3fb-93b068b9d1f6" will pass to kernel 2024-03-08T06:04:29.014 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-08T06:04:29.014 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:04:29.014 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:04:29.015 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:04:29.015 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:04:29.021 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:04:29.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:29.021 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-08T06:04:29.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:29.112 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:29.169 INFO:teuthology.orchestra.run.smithi053.stdout:client4531 2024-03-08T06:04:29.170 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:04:29.170 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:29.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:29.225 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:29.283 INFO:teuthology.orchestra.run.smithi053.stdout:client4531 2024-03-08T06:04:29.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:29.283 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:29.341 INFO:teuthology.orchestra.run.smithi053.stdout:client4531 2024-03-08T06:04:29.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:29.342 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c4e20909-a427-4e44-a3fb-93b068b9d1f6.client4531/status') 2024-03-08T06:04:29.435 INFO:teuthology.orchestra.run.smithi053.stdout:instance: client.4531 (3)192.168.0.1:0/3054772649 2024-03-08T06:04:29.435 INFO:teuthology.orchestra.run.smithi053.stdout:blocklisted: false 2024-03-08T06:04:29.435 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2024-03-08T06:04:29.435 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2024-03-08T06:04:29.435 INFO:teuthology.orchestra.run.smithi053.stderr:69 bytes copied, 7.4721e-05 s, 923 kB/s 2024-03-08T06:04:29.440 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-03-08T06:04:29.440 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T06:04:29.440 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-08T06:04:29.440 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:04:29.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.659+0000 7f96da600640 1 Processor -- start 2024-03-08T06:04:29.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.659+0000 7f96da600640 1 -- start start 2024-03-08T06:04:29.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.661+0000 7f96da600640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96d415c7d0 0x7f96d4163160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:29.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.661+0000 7f96da600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96d4071f50 0x7f96d40b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:29.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.661+0000 7f96da600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d41636a0 con 0x7f96d406e280 2024-03-08T06:04:29.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.661+0000 7f96da600640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d406ddf0 con 0x7f96d415c7d0 2024-03-08T06:04:29.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.661+0000 7f96da600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d415cbb0 con 0x7f96d4071f50 2024-03-08T06:04:29.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.661+0000 7f96d3400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96d415c7d0 0x7f96d4163160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:29.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.661+0000 7f96d2a00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f96d406e280 0x7f96d4071840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56724/0 (socket says 172.21.15.40:56724) 2024-03-08T06:04:29.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.661+0000 7f96d2a00640 1 -- 172.21.15.40:0/1241291625 learned_addr learned my addr 172.21.15.40:0/1241291625 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:29.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.661+0000 7f96d3e00640 1 --2- 172.21.15.40:0/1241291625 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96d4071f50 0x7f96d40b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:29.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.662+0000 7f96d3400640 1 -- 172.21.15.40:0/1241291625 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96d4071f50 msgr2=0x7f96d40b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:29.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.662+0000 7f96d3400640 1 --2- 172.21.15.40:0/1241291625 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96d4071f50 0x7f96d40b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:29.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.662+0000 7f96d3400640 1 -- 172.21.15.40:0/1241291625 >> v1:172.21.15.40:6789/0 conn(0x7f96d406e280 legacy=0x7f96d4071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:29.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.662+0000 7f96d3400640 1 -- 172.21.15.40:0/1241291625 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d40b4c40 con 0x7f96d415c7d0 2024-03-08T06:04:29.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.663+0000 7f96d3400640 1 --2- 172.21.15.40:0/1241291625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96d415c7d0 0x7f96d4163160 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f96c40097a0 tx=0x7f96c402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dad499640cb74f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:29.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.663+0000 7f96d2000640 1 -- 172.21.15.40:0/1241291625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4035020 con 0x7f96d415c7d0 2024-03-08T06:04:29.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.664+0000 7f96d2000640 1 -- 172.21.15.40:0/1241291625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c4031810 con 0x7f96d415c7d0 2024-03-08T06:04:29.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.664+0000 7f96d2000640 1 -- 172.21.15.40:0/1241291625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4031be0 con 0x7f96d415c7d0 2024-03-08T06:04:29.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.664+0000 7f96da600640 1 -- 172.21.15.40:0/1241291625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96d415c7d0 msgr2=0x7f96d4163160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:29.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.664+0000 7f96da600640 1 --2- 172.21.15.40:0/1241291625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96d415c7d0 0x7f96d4163160 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f96c40097a0 tx=0x7f96c402b8b0 comp rx=0 tx=0).stop 2024-03-08T06:04:29.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.664+0000 7f96da600640 1 -- 172.21.15.40:0/1241291625 shutdown_connections 2024-03-08T06:04:29.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.664+0000 7f96da600640 1 --2- 172.21.15.40:0/1241291625 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96d4071f50 0x7f96d40b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:29.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.664+0000 7f96da600640 1 --2- 172.21.15.40:0/1241291625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96d415c7d0 0x7f96d4163160 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:29.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.664+0000 7f96da600640 1 -- 172.21.15.40:0/1241291625 >> 172.21.15.40:0/1241291625 conn(0x7f96d406ae30 msgr2=0x7f96d406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:29.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.664+0000 7f96da600640 1 -- 172.21.15.40:0/1241291625 shutdown_connections 2024-03-08T06:04:29.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.664+0000 7f96da600640 1 -- 172.21.15.40:0/1241291625 wait complete. 2024-03-08T06:04:29.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.666+0000 7f96da600640 1 Processor -- start 2024-03-08T06:04:29.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.666+0000 7f96da600640 1 -- start start 2024-03-08T06:04:29.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.667+0000 7f96da600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d406e280 0x7f96d414d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:29.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.667+0000 7f96da600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96d4071f50 0x7f96d4147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:29.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.667+0000 7f96da600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96d415c7d0 0x7f96d4147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:29.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.667+0000 7f96da600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40b46a0 con 0x7f96d4071f50 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.667+0000 7f96da600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d406da80 con 0x7f96d406e280 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.667+0000 7f96da600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40b4820 con 0x7f96d415c7d0 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.667+0000 7f96d2a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96d4071f50 0x7f96d4147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.667+0000 7f96d2a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96d4071f50 0x7f96d4147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58996/0 (socket says 172.21.15.40:58996) 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.667+0000 7f96d2a00640 1 -- 172.21.15.40:0/3677543046 learned_addr learned my addr 172.21.15.40:0/3677543046 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96d3400640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d406e280 0x7f96d414d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96d2a00640 1 -- 172.21.15.40:0/3677543046 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96d415c7d0 msgr2=0x7f96d4147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96d2a00640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96d415c7d0 0x7f96d4147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96d2a00640 1 -- 172.21.15.40:0/3677543046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d406e280 msgr2=0x7f96d414d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96d2a00640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d406e280 0x7f96d414d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96d2a00640 1 -- 172.21.15.40:0/3677543046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d41482a0 con 0x7f96d4071f50 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96d3400640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d406e280 0x7f96d414d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96d2a00640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96d4071f50 0x7f96d4147730 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c400 tx=0x7f96c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96cb400640 1 -- 172.21.15.40:0/3677543046 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0017070 con 0x7f96d4071f50 2024-03-08T06:04:29.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96cb400640 1 -- 172.21.15.40:0/3677543046 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0005ce0 con 0x7f96d4071f50 2024-03-08T06:04:29.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96cb400640 1 -- 172.21.15.40:0/3677543046 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c00072f0 con 0x7f96d4071f50 2024-03-08T06:04:29.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.668+0000 7f96da600640 1 -- 172.21.15.40:0/3677543046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d4148580 con 0x7f96d4071f50 2024-03-08T06:04:29.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.669+0000 7f96da600640 1 -- 172.21.15.40:0/3677543046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d4148b40 con 0x7f96d4071f50 2024-03-08T06:04:29.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.670+0000 7f96da600640 1 -- 172.21.15.40:0/3677543046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d4067cb0 con 0x7f96d4071f50 2024-03-08T06:04:29.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.671+0000 7f96cb400640 1 -- 172.21.15.40:0/3677543046 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96c001a070 con 0x7f96d4071f50 2024-03-08T06:04:29.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.671+0000 7f96cb400640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96b406b4d0 0x7f96b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:29.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.672+0000 7f96cb400640 1 -- 172.21.15.40:0/3677543046 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96c0089600 con 0x7f96d4071f50 2024-03-08T06:04:29.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.672+0000 7f96d3400640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96b406b4d0 0x7f96b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:29.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.672+0000 7f96d3400640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96b406b4d0 0x7f96b406d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f96c4036000 tx=0x7f96c4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.676+0000 7f96cb400640 1 -- 172.21.15.40:0/3677543046 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96d41636a0 con 0x7f96d4071f50 2024-03-08T06:04:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.943+0000 7f96da600640 1 -- 172.21.15.40:0/3677543046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96d406ebd0 con 0x7f96d4071f50 2024-03-08T06:04:29.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.944+0000 7f96cb400640 1 -- 172.21.15.40:0/3677543046 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f96c0058470 con 0x7f96d4071f50 2024-03-08T06:04:29.945 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.946+0000 7f96da600640 1 -- 172.21.15.40:0/3677543046 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96b406b4d0 msgr2=0x7f96b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.946+0000 7f96da600640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96b406b4d0 0x7f96b406d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f96c4036000 tx=0x7f96c4008030 comp rx=0 tx=0).stop 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.946+0000 7f96da600640 1 -- 172.21.15.40:0/3677543046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96d4071f50 msgr2=0x7f96d4147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.946+0000 7f96da600640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96d4071f50 0x7f96d4147730 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c400 tx=0x7f96c000c8c0 comp rx=0 tx=0).stop 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.946+0000 7f96da600640 1 -- 172.21.15.40:0/3677543046 shutdown_connections 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.946+0000 7f96da600640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96b406b4d0 0x7f96b406d990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.946+0000 7f96da600640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96d415c7d0 0x7f96d4147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.946+0000 7f96da600640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96d4071f50 0x7f96d4147730 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.946+0000 7f96da600640 1 --2- 172.21.15.40:0/3677543046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96d406e280 0x7f96d414d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.946+0000 7f96da600640 1 -- 172.21.15.40:0/3677543046 >> 172.21.15.40:0/3677543046 conn(0x7f96d406ae30 msgr2=0x7f96d40ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.947+0000 7f96da600640 1 -- 172.21.15.40:0/3677543046 shutdown_connections 2024-03-08T06:04:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:29.947+0000 7f96da600640 1 -- 172.21.15.40:0/3677543046 wait complete. 2024-03-08T06:04:29.965 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T06:04:29.965 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:04:29.965 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-03-08T06:04:29.965 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-08T06:04:29.965 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-08T06:04:29.965 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T06:04:29.965 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T06:04:29.965 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:04:29.966 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-08T06:04:29.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:29.966 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-03-08T06:04:30.028 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:04:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:04:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:04:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-03-08T06:04:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.028 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-03-08T06:04:30.028 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42544sec preferred_lft 42544sec 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 6a:91:a5:ab:52:6e brd ff:ff:ff:ff:ff:ff 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::d48d:10ff:fe90:b28/64 scope link 2024-03-08T06:04:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.030 INFO:teuthology.orchestra.run.smithi053.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:04:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 6a:91:a5:ab:52:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T06:04:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::6891:a5ff:feab:526e/64 scope link 2024-03-08T06:04:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:30.031 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:04:30.127 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-03-08T06:04:30.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:30.132 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-03-08T06:04:30.187 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-03-08T06:04:30.187 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:04:30.187 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-08T06:04:30.187 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-08T06:04:30.187 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:04:30.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:30.188 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2024-03-08T06:04:30.189 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-03-08T06:04:30.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-08T06:04:30.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-08T06:04:30.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-08T06:04:30.189 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-03-08T06:04:30.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:30.385 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-03-08T06:04:30.439 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-08T06:04:30.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:30.441 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2024-03-08T06:04:30.495 INFO:teuthology.orchestra.run.smithi053.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-08T06:04:30.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:30.496 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2024-03-08T06:04:30.496 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-03-08T06:04:30.496 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-08T06:04:30.496 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-08T06:04:30.496 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-03-08T06:04:30.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:30.642 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-08T06:04:30.734 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:04:30.734 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:04:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:04:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-03-08T06:04:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.735 INFO:teuthology.orchestra.run.smithi053.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:04:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether a6:c6:74:ed:ee:f4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T06:04:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T06:04:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::a4c6:74ff:feed:eef4/64 scope link 2024-03-08T06:04:30.735 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:30.740 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-08T06:04:30.831 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:04:30.831 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:04:30.831 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:04:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-03-08T06:04:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.832 INFO:teuthology.orchestra.run.smithi053.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:04:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether a6:c6:74:ed:ee:f4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T06:04:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T06:04:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::a4c6:74ff:feed:eef4/64 scope link 2024-03-08T06:04:30.832 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:04:30.836 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-08T06:04:30.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:30.836 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2024-03-08T06:04:30.836 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-03-08T06:04:30.836 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-08T06:04:30.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-08T06:04:30.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-08T06:04:30.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-08T06:04:30.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-08T06:04:30.837 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-03-08T06:04:31.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:31.124 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2024-03-08T06:04:31.124 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-03-08T06:04:31.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.1 up 2024-03-08T06:04:31.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.1 master ceph-brx 2024-03-08T06:04:31.124 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-03-08T06:04:31.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:04:31.299 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-08T06:04:31.349 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-08T06:04:31.349 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:04:31.349 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-08T06:04:31.395 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-08T06:04:31.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:31.396 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-08T06:04:31.586 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-08T06:04:31.586 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-08T06:04:31.586 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-03-08T06:04:31.608 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-08T06:04:31.608 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-08T06:04:31.609 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-03-08T06:04:31.609 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:04:31.609 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:04:31.639 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-08T06:04:31.639 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-08T06:04:31.639 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-03-08T06:04:31.639 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:04:31.639 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:04:31.639 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790" 2024-03-08T06:04:31.640 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790:/ 2024-03-08T06:04:31.640 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=c4e20909-a427-4e44-a3fb-93b068b9d1f6" will pass to kernel 2024-03-08T06:04:31.643 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-08T06:04:31.643 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:04:31.643 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:04:31.643 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:04:31.643 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:04:31.649 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:04:31.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:31.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-08T06:04:31.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:31.735 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:31.797 INFO:teuthology.orchestra.run.smithi053.stdout:client4638 2024-03-08T06:04:31.797 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:04:31.798 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:31.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:31.849 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:31.902 INFO:teuthology.orchestra.run.smithi053.stdout:client4638 2024-03-08T06:04:31.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:31.903 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi053.stdout:client4638 2024-03-08T06:04:31.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:31.959 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c4e20909-a427-4e44-a3fb-93b068b9d1f6.client4638/status') 2024-03-08T06:04:32.055 INFO:teuthology.orchestra.run.smithi053.stdout:instance: client.4638 (3)172.21.15.53:0/1721661473 2024-03-08T06:04:32.055 INFO:teuthology.orchestra.run.smithi053.stdout:blocklisted: false 2024-03-08T06:04:32.055 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2024-03-08T06:04:32.055 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2024-03-08T06:04:32.055 INFO:teuthology.orchestra.run.smithi053.stderr:70 bytes copied, 6.9824e-05 s, 1.0 MB/s 2024-03-08T06:04:32.059 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-08T06:04:32.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.307+0000 7f6276c00640 1 Processor -- start 2024-03-08T06:04:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.307+0000 7f6276c00640 1 -- start start 2024-03-08T06:04:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.308+0000 7f6276c00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f627015a770 0x7f627015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.308+0000 7f6276c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f62700cc310 0x7f627006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.308+0000 7f6276c00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627006ec90 con 0x7f627006e870 2024-03-08T06:04:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.308+0000 7f6276c00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62700ccdd0 con 0x7f62700cc310 2024-03-08T06:04:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.308+0000 7f6276c00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627015e8f0 con 0x7f627015a770 2024-03-08T06:04:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.309+0000 7f6274e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f62700cc310 0x7f627006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.309+0000 7f6274e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f62700cc310 0x7f627006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59010/0 (socket says 172.21.15.40:59010) 2024-03-08T06:04:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.309+0000 7f6274e00640 1 -- 172.21.15.40:0/92509463 learned_addr learned my addr 172.21.15.40:0/92509463 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.309+0000 7f6275800640 1 --2- 172.21.15.40:0/92509463 >> v2:172.21.15.53:3300/0 conn(0x7f627015a770 0x7f627015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.309+0000 7f6274e00640 1 -- 172.21.15.40:0/92509463 >> v2:172.21.15.53:3300/0 conn(0x7f627015a770 msgr2=0x7f627015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.309+0000 7f6274e00640 1 --2- 172.21.15.40:0/92509463 >> v2:172.21.15.53:3300/0 conn(0x7f627015a770 0x7f627015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.309+0000 7f6274e00640 1 -- 172.21.15.40:0/92509463 >> v1:172.21.15.53:6789/0 conn(0x7f627006e870 legacy=0x7f62700716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.309+0000 7f6274e00640 1 -- 172.21.15.40:0/92509463 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62700cd020 con 0x7f62700cc310 2024-03-08T06:04:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.310+0000 7f6274e00640 1 --2- 172.21.15.40:0/92509463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62700cc310 0x7f627006e330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f627006f4d0 tx=0x7f625c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6b7d5f2c3c3817 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.311+0000 7f626fe00640 1 -- 172.21.15.40:0/92509463 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c035020 con 0x7f62700cc310 2024-03-08T06:04:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.311+0000 7f626fe00640 1 -- 172.21.15.40:0/92509463 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c031870 con 0x7f62700cc310 2024-03-08T06:04:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.311+0000 7f626fe00640 1 -- 172.21.15.40:0/92509463 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c031c40 con 0x7f62700cc310 2024-03-08T06:04:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.311+0000 7f6276c00640 1 -- 172.21.15.40:0/92509463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62700cc310 msgr2=0x7f627006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.311+0000 7f6276c00640 1 --2- 172.21.15.40:0/92509463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62700cc310 0x7f627006e330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f627006f4d0 tx=0x7f625c02b910 comp rx=0 tx=0).stop 2024-03-08T06:04:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.311+0000 7f6276c00640 1 -- 172.21.15.40:0/92509463 shutdown_connections 2024-03-08T06:04:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.311+0000 7f6276c00640 1 --2- 172.21.15.40:0/92509463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62700cc310 0x7f627006e330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.311+0000 7f6276c00640 1 --2- 172.21.15.40:0/92509463 >> v2:172.21.15.53:3300/0 conn(0x7f627015a770 0x7f627015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.311+0000 7f6276c00640 1 -- 172.21.15.40:0/92509463 >> 172.21.15.40:0/92509463 conn(0x7f62700b1850 msgr2=0x7f62700b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.311+0000 7f6276c00640 1 -- 172.21.15.40:0/92509463 shutdown_connections 2024-03-08T06:04:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.311+0000 7f6276c00640 1 -- 172.21.15.40:0/92509463 wait complete. 2024-03-08T06:04:32.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.313+0000 7f6276c00640 1 Processor -- start 2024-03-08T06:04:32.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.313+0000 7f6276c00640 1 -- start start 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.314+0000 7f6276c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f627006e870 0x7f627014a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.314+0000 7f6276c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f62700cc310 0x7f62701e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.314+0000 7f6276c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f627015a770 0x7f62701e5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.314+0000 7f6276c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627015e580 con 0x7f627015a770 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.314+0000 7f6276c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627015f2c0 con 0x7f627006e870 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.314+0000 7f6276c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62700b4e00 con 0x7f62700cc310 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.314+0000 7f6276200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f627015a770 0x7f62701e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.314+0000 7f6276200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f627015a770 0x7f62701e5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59020/0 (socket says 172.21.15.40:59020) 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.314+0000 7f6276200640 1 -- 172.21.15.40:0/1934400674 learned_addr learned my addr 172.21.15.40:0/1934400674 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.315+0000 7f6276200640 1 -- 172.21.15.40:0/1934400674 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f62700cc310 msgr2=0x7f62701e4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.315+0000 7f6276200640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f62700cc310 0x7f62701e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.315+0000 7f6276200640 1 -- 172.21.15.40:0/1934400674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f627006e870 msgr2=0x7f627014a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.315+0000 7f6276200640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f627006e870 0x7f627014a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.315+0000 7f6276200640 1 -- 172.21.15.40:0/1934400674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6270158c70 con 0x7f627015a770 2024-03-08T06:04:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.315+0000 7f6276200640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f627015a770 0x7f62701e5400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f626800c780 tx=0x7f626800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.315+0000 7f626d600640 1 -- 172.21.15.40:0/1934400674 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268017070 con 0x7f627015a770 2024-03-08T06:04:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.315+0000 7f626d600640 1 -- 172.21.15.40:0/1934400674 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6268003da0 con 0x7f627015a770 2024-03-08T06:04:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.315+0000 7f626d600640 1 -- 172.21.15.40:0/1934400674 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62680052b0 con 0x7f627015a770 2024-03-08T06:04:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.315+0000 7f6276c00640 1 -- 172.21.15.40:0/1934400674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6270158f50 con 0x7f627015a770 2024-03-08T06:04:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.316+0000 7f6276c00640 1 -- 172.21.15.40:0/1934400674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6270159510 con 0x7f627015a770 2024-03-08T06:04:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.317+0000 7f6276c00640 1 -- 172.21.15.40:0/1934400674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6270075100 con 0x7f627015a770 2024-03-08T06:04:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.318+0000 7f626d600640 1 -- 172.21.15.40:0/1934400674 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f626801a070 con 0x7f627015a770 2024-03-08T06:04:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.319+0000 7f626d600640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f625006b4d0 0x7f625006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.319+0000 7f626d600640 1 -- 172.21.15.40:0/1934400674 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6268089920 con 0x7f627015a770 2024-03-08T06:04:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.319+0000 7f6275800640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f625006b4d0 0x7f625006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.320+0000 7f6275800640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f625006b4d0 0x7f625006d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6264006fd0 tx=0x7f6264008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.324+0000 7f626d600640 1 -- 172.21.15.40:0/1934400674 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6268058700 con 0x7f627015a770 2024-03-08T06:04:32.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.526+0000 7f6276c00640 1 -- 172.21.15.40:0/1934400674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f627015da60 con 0x7f627015a770 2024-03-08T06:04:32.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.528+0000 7f626d600640 1 -- 172.21.15.40:0/1934400674 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f6268058000 con 0x7f627015a770 2024-03-08T06:04:32.530 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:32.530 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:32.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 -- 172.21.15.40:0/1934400674 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f625006b4d0 msgr2=0x7f625006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f625006b4d0 0x7f625006d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6264006fd0 tx=0x7f6264008040 comp rx=0 tx=0).stop 2024-03-08T06:04:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 -- 172.21.15.40:0/1934400674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f627015a770 msgr2=0x7f62701e5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f627015a770 0x7f62701e5400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f626800c780 tx=0x7f626800cc40 comp rx=0 tx=0).stop 2024-03-08T06:04:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 -- 172.21.15.40:0/1934400674 shutdown_connections 2024-03-08T06:04:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f625006b4d0 0x7f625006d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f627015a770 0x7f62701e5400 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f62700cc310 0x7f62701e4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 --2- 172.21.15.40:0/1934400674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f627006e870 0x7f627014a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 -- 172.21.15.40:0/1934400674 >> 172.21.15.40:0/1934400674 conn(0x7f62700b1850 msgr2=0x7f627006d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 -- 172.21.15.40:0/1934400674 shutdown_connections 2024-03-08T06:04:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.531+0000 7f6276c00640 1 -- 172.21.15.40:0/1934400674 wait complete. 2024-03-08T06:04:32.552 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:32.553 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.772+0000 7fe3e4a00640 1 Processor -- start 2024-03-08T06:04:32.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.772+0000 7fe3e4a00640 1 -- start start 2024-03-08T06:04:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.773+0000 7fe3e4a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3e0161930 0x7fe3e00b9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.773+0000 7fe3e4a00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e015e240 con 0x7fe3e015de60 2024-03-08T06:04:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.773+0000 7fe3e4a00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e0161d80 con 0x7fe3e00cf030 2024-03-08T06:04:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.773+0000 7fe3e4a00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e00b9b70 con 0x7fe3e0161930 2024-03-08T06:04:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.774+0000 7fe3df400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe3e015de60 0x7fe3e00ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56732/0 (socket says 172.21.15.40:56732) 2024-03-08T06:04:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.774+0000 7fe3df400640 1 -- 172.21.15.40:0/3072691547 learned_addr learned my addr 172.21.15.40:0/3072691547 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.774+0000 7fe3dfe00640 1 --2- 172.21.15.40:0/3072691547 >> v2:172.21.15.40:3300/0 conn(0x7fe3e0161930 0x7fe3e00b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.774+0000 7fe3dfe00640 1 -- 172.21.15.40:0/3072691547 >> v1:172.21.15.53:6789/0 conn(0x7fe3e00cf030 legacy=0x7fe3e00d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.774+0000 7fe3dfe00640 1 -- 172.21.15.40:0/3072691547 >> v1:172.21.15.40:6789/0 conn(0x7fe3e015de60 legacy=0x7fe3e00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.774+0000 7fe3dfe00640 1 -- 172.21.15.40:0/3072691547 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e0162030 con 0x7fe3e0161930 2024-03-08T06:04:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.774+0000 7fe3dea00640 1 -- 172.21.15.40:0/3072691547 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479239991 0 0) 0x7fe3e015e240 con 0x7fe3e015de60 2024-03-08T06:04:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.775+0000 7fe3dfe00640 1 --2- 172.21.15.40:0/3072691547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e0161930 0x7fe3e00b9630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009d90 tx=0x7fe3d802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04648a8eb834473 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.776+0000 7fe3dea00640 1 -- 172.21.15.40:0/3072691547 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8039070 con 0x7fe3e0161930 2024-03-08T06:04:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.776+0000 7fe3dea00640 1 -- 172.21.15.40:0/3072691547 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d802cc60 con 0x7fe3e0161930 2024-03-08T06:04:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.776+0000 7fe3dea00640 1 -- 172.21.15.40:0/3072691547 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8034660 con 0x7fe3e0161930 2024-03-08T06:04:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.776+0000 7fe3e4a00640 1 -- 172.21.15.40:0/3072691547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e0161930 msgr2=0x7fe3e00b9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.776+0000 7fe3e4a00640 1 --2- 172.21.15.40:0/3072691547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e0161930 0x7fe3e00b9630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009d90 tx=0x7fe3d802beb0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.776+0000 7fe3e4a00640 1 -- 172.21.15.40:0/3072691547 shutdown_connections 2024-03-08T06:04:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.776+0000 7fe3e4a00640 1 --2- 172.21.15.40:0/3072691547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e0161930 0x7fe3e00b9630 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.776+0000 7fe3e4a00640 1 -- 172.21.15.40:0/3072691547 >> 172.21.15.40:0/3072691547 conn(0x7fe3e006c500 msgr2=0x7fe3e006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.776+0000 7fe3e4a00640 1 -- 172.21.15.40:0/3072691547 shutdown_connections 2024-03-08T06:04:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.776+0000 7fe3e4a00640 1 -- 172.21.15.40:0/3072691547 wait complete. 2024-03-08T06:04:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.778+0000 7fe3e4a00640 1 Processor -- start 2024-03-08T06:04:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.778+0000 7fe3e4a00640 1 -- start start 2024-03-08T06:04:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.779+0000 7fe3e4a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e00cf030 0x7fe3e01573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.779+0000 7fe3e4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e015de60 0x7fe3e015ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.779+0000 7fe3e4a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe3e0161930 0x7fe3e00b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.779+0000 7fe3e4a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00d20c0 con 0x7fe3e00cf030 2024-03-08T06:04:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.779+0000 7fe3e4a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e01625f0 con 0x7fe3e015de60 2024-03-08T06:04:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.779+0000 7fe3e4a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00d2cf0 con 0x7fe3e0161930 2024-03-08T06:04:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.779+0000 7fe3df400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e00cf030 0x7fe3e01573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3df400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e00cf030 0x7fe3e01573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59038/0 (socket says 172.21.15.40:59038) 2024-03-08T06:04:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3df400640 1 -- 172.21.15.40:0/2860550236 learned_addr learned my addr 172.21.15.40:0/2860550236 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3d7e00640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e015de60 0x7fe3e015ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3dfe00640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe3e0161930 0x7fe3e00b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3df400640 1 -- 172.21.15.40:0/2860550236 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe3e0161930 msgr2=0x7fe3e00b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3df400640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe3e0161930 0x7fe3e00b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3df400640 1 -- 172.21.15.40:0/2860550236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e015de60 msgr2=0x7fe3e015ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3df400640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e015de60 0x7fe3e015ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3df400640 1 -- 172.21.15.40:0/2860550236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e01578f0 con 0x7fe3e00cf030 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3dfe00640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe3e0161930 0x7fe3e00b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3df400640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e00cf030 0x7fe3e01573b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c4d0 tx=0x7fe3d000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.780+0000 7fe3d7400640 1 -- 172.21.15.40:0/2860550236 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0018070 con 0x7fe3e00cf030 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.781+0000 7fe3d7400640 1 -- 172.21.15.40:0/2860550236 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d0007500 con 0x7fe3e00cf030 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.781+0000 7fe3e4a00640 1 -- 172.21.15.40:0/2860550236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e00b6350 con 0x7fe3e00cf030 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.781+0000 7fe3d7400640 1 -- 172.21.15.40:0/2860550236 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0004a20 con 0x7fe3e00cf030 2024-03-08T06:04:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.781+0000 7fe3e4a00640 1 -- 172.21.15.40:0/2860550236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e00cbf50 con 0x7fe3e00cf030 2024-03-08T06:04:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.782+0000 7fe3e4a00640 1 -- 172.21.15.40:0/2860550236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e0075090 con 0x7fe3e00cf030 2024-03-08T06:04:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.783+0000 7fe3d7400640 1 -- 172.21.15.40:0/2860550236 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3d001b070 con 0x7fe3e00cf030 2024-03-08T06:04:32.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.784+0000 7fe3d7400640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe3bc06b4d0 0x7fe3bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:32.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.784+0000 7fe3d7400640 1 -- 172.21.15.40:0/2860550236 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3d00895b0 con 0x7fe3e00cf030 2024-03-08T06:04:32.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.784+0000 7fe3d7e00640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe3bc06b4d0 0x7fe3bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.785+0000 7fe3d7e00640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe3bc06b4d0 0x7fe3bc06d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00cdc90 tx=0x7fe3c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:32.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.788+0000 7fe3d7400640 1 -- 172.21.15.40:0/2860550236 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3d0058390 con 0x7fe3e00cf030 2024-03-08T06:04:32.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.988+0000 7fe3e4a00640 1 -- 172.21.15.40:0/2860550236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3e00d17f0 con 0x7fe3e00cf030 2024-03-08T06:04:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.989+0000 7fe3d7400640 1 -- 172.21.15.40:0/2860550236 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3d0057c90 con 0x7fe3e00cf030 2024-03-08T06:04:32.990 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 -- 172.21.15.40:0/2860550236 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe3bc06b4d0 msgr2=0x7fe3bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe3bc06b4d0 0x7fe3bc06d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00cdc90 tx=0x7fe3c8009040 comp rx=0 tx=0).stop 2024-03-08T06:04:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 -- 172.21.15.40:0/2860550236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e00cf030 msgr2=0x7fe3e01573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e00cf030 0x7fe3e01573b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c4d0 tx=0x7fe3d000c990 comp rx=0 tx=0).stop 2024-03-08T06:04:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 -- 172.21.15.40:0/2860550236 shutdown_connections 2024-03-08T06:04:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe3bc06b4d0 0x7fe3bc06d990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe3e0161930 0x7fe3e00b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e015de60 0x7fe3e015ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 --2- 172.21.15.40:0/2860550236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3e00cf030 0x7fe3e01573b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 -- 172.21.15.40:0/2860550236 >> 172.21.15.40:0/2860550236 conn(0x7fe3e006c500 msgr2=0x7fe3e00c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 -- 172.21.15.40:0/2860550236 shutdown_connections 2024-03-08T06:04:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:32.992+0000 7fe3e4a00640 1 -- 172.21.15.40:0/2860550236 wait complete. 2024-03-08T06:04:33.013 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:33.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.238+0000 7f38c9600640 1 Processor -- start 2024-03-08T06:04:33.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.238+0000 7f38c9600640 1 -- start start 2024-03-08T06:04:33.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.240+0000 7f38c9600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c4071bc0 0x7f38c406e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.240+0000 7f38c9600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c40b3120 con 0x7f38c41623a0 2024-03-08T06:04:33.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.240+0000 7f38c9600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c40b3400 con 0x7f38c4071fa0 2024-03-08T06:04:33.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.240+0000 7f38c9600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40b36e0 con 0x7f38c4071bc0 2024-03-08T06:04:33.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.240+0000 7f38c3400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f38c41623a0 0x7f38c40ce1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56746/0 (socket says 172.21.15.40:56746) 2024-03-08T06:04:33.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.240+0000 7f38c3400640 1 -- 172.21.15.40:0/916704087 learned_addr learned my addr 172.21.15.40:0/916704087 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:33.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.240+0000 7f38c2a00640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c4071bc0 0x7f38c406e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893540617 0 0) 0x7f38c40b3120 con 0x7f38c41623a0 2024-03-08T06:04:33.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38a8003520 con 0x7f38c41623a0 2024-03-08T06:04:33.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038516653 0 0) 0x7f38c40b3400 con 0x7f38c4071fa0 2024-03-08T06:04:33.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38c40b3120 con 0x7f38c4071fa0 2024-03-08T06:04:33.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1943861259 0 0) 0x7f38a8003520 con 0x7f38c41623a0 2024-03-08T06:04:33.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c4071bc0 msgr2=0x7f38c406e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:33.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c4071bc0 0x7f38c406e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 >> v1:172.21.15.53:6789/0 conn(0x7f38c4071fa0 legacy=0x7f38c4161c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c40b39c0 con 0x7f38c41623a0 2024-03-08T06:04:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f38bc001670 con 0x7f38c41623a0 2024-03-08T06:04:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 >> v1:172.21.15.40:6789/0 conn(0x7f38c41623a0 legacy=0x7f38c40ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c4071bc0 0x7f38a8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8004120 0x7f38a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8004d40 0x7f38a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40b36e0 con 0x7f38a8004d40 2024-03-08T06:04:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a800bb00 con 0x7f38a8004120 2024-03-08T06:04:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.241+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a800bd00 con 0x7f38c4071bc0 2024-03-08T06:04:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.242+0000 7f38c3400640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8004d40 0x7f38a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.242+0000 7f38c2a00640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c4071bc0 0x7f38a8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.242+0000 7f38ba000640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8004120 0x7f38a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.242+0000 7f38c3400640 1 -- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c4071bc0 msgr2=0x7f38a8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.242+0000 7f38c3400640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c4071bc0 0x7f38a8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.242+0000 7f38c3400640 1 -- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8004120 msgr2=0x7f38a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.242+0000 7f38c3400640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8004120 0x7f38a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.242+0000 7f38c3400640 1 -- 172.21.15.40:0/916704087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a8003520 con 0x7f38a8004d40 2024-03-08T06:04:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.242+0000 7f38ba000640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8004120 0x7f38a8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.243+0000 7f38c3400640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8004d40 0x7f38a80051c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f38bc007fc0 tx=0x7f38a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f927074300957c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.244+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc007be0 con 0x7f38a8004d40 2024-03-08T06:04:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.244+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc030070 con 0x7f38a8004d40 2024-03-08T06:04:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.244+0000 7f38c2000640 1 -- 172.21.15.40:0/916704087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc005d30 con 0x7f38a8004d40 2024-03-08T06:04:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.244+0000 7f38c9600640 1 -- 172.21.15.40:0/916704087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8004d40 msgr2=0x7f38a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.244+0000 7f38c9600640 1 --2- 172.21.15.40:0/916704087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8004d40 0x7f38a80051c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f38bc007fc0 tx=0x7f38a8005830 comp rx=0 tx=0).stop 2024-03-08T06:04:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.244+0000 7f38c3400640 1 -- 172.21.15.40:0/916704087 reap_dead start 2024-03-08T06:04:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.244+0000 7f38c9600640 1 -- 172.21.15.40:0/916704087 shutdown_connections 2024-03-08T06:04:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.244+0000 7f38c9600640 1 -- 172.21.15.40:0/916704087 >> 172.21.15.40:0/916704087 conn(0x7f38c406ae30 msgr2=0x7f38c406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.244+0000 7f38c9600640 1 -- 172.21.15.40:0/916704087 shutdown_connections 2024-03-08T06:04:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.244+0000 7f38c9600640 1 -- 172.21.15.40:0/916704087 wait complete. 2024-03-08T06:04:33.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.246+0000 7f38c9600640 1 Processor -- start 2024-03-08T06:04:33.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.246+0000 7f38c9600640 1 -- start start 2024-03-08T06:04:33.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.247+0000 7f38c9600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8004d40 0x7f38c414b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.247+0000 7f38c9600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38c414dc20 0x7f38c414c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.247+0000 7f38c9600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c414a660 0x7f38c40aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.247+0000 7f38c9600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40b2e30 con 0x7f38c414dc20 2024-03-08T06:04:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.247+0000 7f38c9600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40b2cb0 con 0x7f38a8004d40 2024-03-08T06:04:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.247+0000 7f38c9600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c406e190 con 0x7f38c414a660 2024-03-08T06:04:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.247+0000 7f38ba000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38c414dc20 0x7f38c414c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.248+0000 7f38ba000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38c414dc20 0x7f38c414c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59060/0 (socket says 172.21.15.40:59060) 2024-03-08T06:04:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.248+0000 7f38ba000640 1 -- 172.21.15.40:0/985878714 learned_addr learned my addr 172.21.15.40:0/985878714 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.248+0000 7f38c2a00640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8004d40 0x7f38c414b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.248+0000 7f38c3400640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c414a660 0x7f38c40aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.248+0000 7f38ba000640 1 -- 172.21.15.40:0/985878714 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c414a660 msgr2=0x7f38c40aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.248+0000 7f38ba000640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c414a660 0x7f38c40aca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.248+0000 7f38ba000640 1 -- 172.21.15.40:0/985878714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8004d40 msgr2=0x7f38c414b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.248+0000 7f38ba000640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8004d40 0x7f38c414b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.248+0000 7f38ba000640 1 -- 172.21.15.40:0/985878714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c414e0a0 con 0x7f38c414dc20 2024-03-08T06:04:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.248+0000 7f38ba000640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38c414dc20 0x7f38c414c8c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00d7d0 tx=0x7f38ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.249+0000 7f38bb400640 1 -- 172.21.15.40:0/985878714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac016070 con 0x7f38c414dc20 2024-03-08T06:04:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.249+0000 7f38bb400640 1 -- 172.21.15.40:0/985878714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac00bac0 con 0x7f38c414dc20 2024-03-08T06:04:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.249+0000 7f38c9600640 1 -- 172.21.15.40:0/985878714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c40ad560 con 0x7f38c414dc20 2024-03-08T06:04:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.249+0000 7f38bb400640 1 -- 172.21.15.40:0/985878714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac007020 con 0x7f38c414dc20 2024-03-08T06:04:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.249+0000 7f38c9600640 1 -- 172.21.15.40:0/985878714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c414bb00 con 0x7f38c414dc20 2024-03-08T06:04:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.250+0000 7f38c9600640 1 -- 172.21.15.40:0/985878714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c4067cb0 con 0x7f38c414dc20 2024-03-08T06:04:33.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.251+0000 7f38bb400640 1 -- 172.21.15.40:0/985878714 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38ac0071c0 con 0x7f38c414dc20 2024-03-08T06:04:33.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.252+0000 7f38bb400640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f38a006b500 0x7f38a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.252+0000 7f38bb400640 1 -- 172.21.15.40:0/985878714 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38ac08a240 con 0x7f38c414dc20 2024-03-08T06:04:33.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.253+0000 7f38c2a00640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f38a006b500 0x7f38a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.253+0000 7f38c2a00640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f38a006b500 0x7f38a006d9c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38b4007520 tx=0x7f38b400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.257+0000 7f38bb400640 1 -- 172.21.15.40:0/985878714 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38ac058f90 con 0x7f38c414dc20 2024-03-08T06:04:33.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.454+0000 7f38c9600640 1 -- 172.21.15.40:0/985878714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38c4165670 con 0x7f38c414dc20 2024-03-08T06:04:33.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.455+0000 7f38bb400640 1 -- 172.21.15.40:0/985878714 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38ac058890 con 0x7f38c414dc20 2024-03-08T06:04:33.456 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:33.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 -- 172.21.15.40:0/985878714 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f38a006b500 msgr2=0x7f38a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f38a006b500 0x7f38a006d9c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38b4007520 tx=0x7f38b400b040 comp rx=0 tx=0).stop 2024-03-08T06:04:33.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 -- 172.21.15.40:0/985878714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38c414dc20 msgr2=0x7f38c414c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38c414dc20 0x7f38c414c8c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00d7d0 tx=0x7f38ac00dc90 comp rx=0 tx=0).stop 2024-03-08T06:04:33.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 -- 172.21.15.40:0/985878714 shutdown_connections 2024-03-08T06:04:33.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f38a006b500 0x7f38a006d9c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38c414a660 0x7f38c40aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38c414dc20 0x7f38c414c8c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 --2- 172.21.15.40:0/985878714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8004d40 0x7f38c414b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 -- 172.21.15.40:0/985878714 >> 172.21.15.40:0/985878714 conn(0x7f38c406ae30 msgr2=0x7f38c406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:33.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 -- 172.21.15.40:0/985878714 shutdown_connections 2024-03-08T06:04:33.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.458+0000 7f38c9600640 1 -- 172.21.15.40:0/985878714 wait complete. 2024-03-08T06:04:33.479 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:33.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.706+0000 7f1905200640 1 Processor -- start 2024-03-08T06:04:33.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.706+0000 7f1905200640 1 -- start start 2024-03-08T06:04:33.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.707+0000 7f1905200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f190015de60 0x7f1900161930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.707+0000 7f1905200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19000b9630 0x7f19000ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.707+0000 7f1905200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190015e240 con 0x7f19000cee60 2024-03-08T06:04:33.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.707+0000 7f1905200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1900162030 con 0x7f19000b9630 2024-03-08T06:04:33.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.707+0000 7f1905200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1900162310 con 0x7f190015de60 2024-03-08T06:04:33.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.708+0000 7f18fea00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19000b9630 0x7f19000ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.708+0000 7f18ff400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f190015de60 0x7f1900161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.708+0000 7f18ff400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f190015de60 0x7f1900161930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:54140/0 (socket says 172.21.15.40:54140) 2024-03-08T06:04:33.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.708+0000 7f18ff400640 1 -- 172.21.15.40:0/3919259030 learned_addr learned my addr 172.21.15.40:0/3919259030 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.709+0000 7f18ff400640 1 -- 172.21.15.40:0/3919259030 >> v2:172.21.15.53:3300/0 conn(0x7f19000b9630 msgr2=0x7f19000ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.709+0000 7f18ff400640 1 --2- 172.21.15.40:0/3919259030 >> v2:172.21.15.53:3300/0 conn(0x7f19000b9630 0x7f19000ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.709+0000 7f18ff400640 1 -- 172.21.15.40:0/3919259030 >> v1:172.21.15.53:6789/0 conn(0x7f19000cee60 legacy=0x7f19000b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.709+0000 7f18ff400640 1 -- 172.21.15.40:0/3919259030 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19001625f0 con 0x7f190015de60 2024-03-08T06:04:33.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.710+0000 7f18ff400640 1 --2- 172.21.15.40:0/3919259030 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f190015de60 0x7f1900161930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009e50 tx=0x7f18f002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1a9cb83be3d2a2e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:33.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.711+0000 7f18fe000640 1 -- 172.21.15.40:0/3919259030 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f002cb00 con 0x7f190015de60 2024-03-08T06:04:33.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.711+0000 7f18fe000640 1 -- 172.21.15.40:0/3919259030 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f002cca0 con 0x7f190015de60 2024-03-08T06:04:33.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.711+0000 7f18fe000640 1 -- 172.21.15.40:0/3919259030 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0034610 con 0x7f190015de60 2024-03-08T06:04:33.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.712+0000 7f1905200640 1 -- 172.21.15.40:0/3919259030 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f190015de60 msgr2=0x7f1900161930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.712+0000 7f1905200640 1 --2- 172.21.15.40:0/3919259030 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f190015de60 0x7f1900161930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009e50 tx=0x7f18f002bf60 comp rx=0 tx=0).stop 2024-03-08T06:04:33.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.712+0000 7f1905200640 1 -- 172.21.15.40:0/3919259030 shutdown_connections 2024-03-08T06:04:33.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.712+0000 7f1905200640 1 --2- 172.21.15.40:0/3919259030 >> v2:172.21.15.53:3300/0 conn(0x7f19000b9630 0x7f19000ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.712+0000 7f1905200640 1 --2- 172.21.15.40:0/3919259030 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f190015de60 0x7f1900161930 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.712+0000 7f1905200640 1 -- 172.21.15.40:0/3919259030 >> 172.21.15.40:0/3919259030 conn(0x7f190006c500 msgr2=0x7f190006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:33.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.712+0000 7f1905200640 1 -- 172.21.15.40:0/3919259030 shutdown_connections 2024-03-08T06:04:33.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.712+0000 7f1905200640 1 -- 172.21.15.40:0/3919259030 wait complete. 2024-03-08T06:04:33.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.713+0000 7f1905200640 1 Processor -- start 2024-03-08T06:04:33.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.714+0000 7f1905200640 1 -- start start 2024-03-08T06:04:33.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.714+0000 7f1905200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19000b9630 0x7f1900155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f1905200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19000cee60 0x7f19000c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f1905200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f190015de60 0x7f19000c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f1905200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19000d35f0 con 0x7f19000b9630 2024-03-08T06:04:33.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f1905200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19000b9a90 con 0x7f19000cee60 2024-03-08T06:04:33.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f1905200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19000d3770 con 0x7f190015de60 2024-03-08T06:04:33.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f18ff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19000b9630 0x7f1900155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f18ff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19000b9630 0x7f1900155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59072/0 (socket says 172.21.15.40:59072) 2024-03-08T06:04:33.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f18ff400640 1 -- 172.21.15.40:0/84306445 learned_addr learned my addr 172.21.15.40:0/84306445 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:33.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f18fea00640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19000cee60 0x7f19000c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f18ff400640 1 -- 172.21.15.40:0/84306445 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f190015de60 msgr2=0x7f19000c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f18ff400640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f190015de60 0x7f19000c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f18ff400640 1 -- 172.21.15.40:0/84306445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19000cee60 msgr2=0x7f19000c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f18ff400640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19000cee60 0x7f19000c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f18ff400640 1 -- 172.21.15.40:0/84306445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1900156850 con 0x7f19000b9630 2024-03-08T06:04:33.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.715+0000 7f18fea00640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19000cee60 0x7f19000c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:33.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.716+0000 7f18ff400640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19000b9630 0x7f1900155ba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009e50 tx=0x7f18f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:33.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.716+0000 7f18f7400640 1 -- 172.21.15.40:0/84306445 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0043020 con 0x7f19000b9630 2024-03-08T06:04:33.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.716+0000 7f18f7400640 1 -- 172.21.15.40:0/84306445 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f002ce50 con 0x7f19000b9630 2024-03-08T06:04:33.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.716+0000 7f1905200640 1 -- 172.21.15.40:0/84306445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1900159100 con 0x7f19000b9630 2024-03-08T06:04:33.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.716+0000 7f18f7400640 1 -- 172.21.15.40:0/84306445 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0009510 con 0x7f19000b9630 2024-03-08T06:04:33.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.716+0000 7f1905200640 1 -- 172.21.15.40:0/84306445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19000b4760 con 0x7f19000b9630 2024-03-08T06:04:33.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.718+0000 7f1905200640 1 -- 172.21.15.40:0/84306445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19000b4550 con 0x7f19000b9630 2024-03-08T06:04:33.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.718+0000 7f18f7400640 1 -- 172.21.15.40:0/84306445 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18f0046020 con 0x7f19000b9630 2024-03-08T06:04:33.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.719+0000 7f18f7400640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f18dc06b500 0x7f18dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:33.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.719+0000 7f18f7400640 1 -- 172.21.15.40:0/84306445 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18f00ac5b0 con 0x7f19000b9630 2024-03-08T06:04:33.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.719+0000 7f18fea00640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f18dc06b500 0x7f18dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:33.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.720+0000 7f18fea00640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f18dc06b500 0x7f18dc06d9c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1900160cb0 tx=0x7f18e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:33.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.724+0000 7f18f7400640 1 -- 172.21.15.40:0/84306445 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18f004d070 con 0x7f19000b9630 2024-03-08T06:04:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.924+0000 7f1905200640 1 -- 172.21.15.40:0/84306445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1900075090 con 0x7f19000b9630 2024-03-08T06:04:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.924+0000 7f18f7400640 1 -- 172.21.15.40:0/84306445 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f18f007b3e0 con 0x7f19000b9630 2024-03-08T06:04:33.926 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 -- 172.21.15.40:0/84306445 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f18dc06b500 msgr2=0x7f18dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f18dc06b500 0x7f18dc06d9c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1900160cb0 tx=0x7f18e8008040 comp rx=0 tx=0).stop 2024-03-08T06:04:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 -- 172.21.15.40:0/84306445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19000b9630 msgr2=0x7f1900155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19000b9630 0x7f1900155ba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009e50 tx=0x7f18f0004a40 comp rx=0 tx=0).stop 2024-03-08T06:04:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 -- 172.21.15.40:0/84306445 shutdown_connections 2024-03-08T06:04:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f18dc06b500 0x7f18dc06d9c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f190015de60 0x7f19000c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19000cee60 0x7f19000c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 --2- 172.21.15.40:0/84306445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19000b9630 0x7f1900155ba0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 -- 172.21.15.40:0/84306445 >> 172.21.15.40:0/84306445 conn(0x7f190006c500 msgr2=0x7f19000b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 -- 172.21.15.40:0/84306445 shutdown_connections 2024-03-08T06:04:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:33.927+0000 7f1905200640 1 -- 172.21.15.40:0/84306445 wait complete. 2024-03-08T06:04:33.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.169+0000 7f4a3d800640 1 Processor -- start 2024-03-08T06:04:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.169+0000 7f4a3d800640 1 -- start start 2024-03-08T06:04:34.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.171+0000 7f4a3d800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4a381445d0 0x7f4a3806deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.171+0000 7f4a3d800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a38144d50 0x7f4a38071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.171+0000 7f4a3d800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a38071dc0 con 0x7f4a380719b0 2024-03-08T06:04:34.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.171+0000 7f4a3d800640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3806e3f0 con 0x7f4a38144d50 2024-03-08T06:04:34.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.171+0000 7f4a3d800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3806ebf0 con 0x7f4a381445d0 2024-03-08T06:04:34.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.171+0000 7f4a37400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a38144d50 0x7f4a38071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.171+0000 7f4a37400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a38144d50 0x7f4a38071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59078/0 (socket says 172.21.15.40:59078) 2024-03-08T06:04:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.171+0000 7f4a37400640 1 -- 172.21.15.40:0/1605915874 learned_addr learned my addr 172.21.15.40:0/1605915874 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.172+0000 7f4a37e00640 1 --2- 172.21.15.40:0/1605915874 >> v2:172.21.15.53:3300/0 conn(0x7f4a381445d0 0x7f4a3806deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.172+0000 7f4a37400640 1 -- 172.21.15.40:0/1605915874 >> v2:172.21.15.53:3300/0 conn(0x7f4a381445d0 msgr2=0x7f4a3806deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.172+0000 7f4a37400640 1 --2- 172.21.15.40:0/1605915874 >> v2:172.21.15.53:3300/0 conn(0x7f4a381445d0 0x7f4a3806deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.172+0000 7f4a37400640 1 -- 172.21.15.40:0/1605915874 >> v1:172.21.15.53:6789/0 conn(0x7f4a380719b0 legacy=0x7f4a3815f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.172+0000 7f4a37400640 1 -- 172.21.15.40:0/1605915874 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3806eed0 con 0x7f4a38144d50 2024-03-08T06:04:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.172+0000 7f4a37e00640 1 --2- 172.21.15.40:0/1605915874 >> v2:172.21.15.53:3300/0 conn(0x7f4a381445d0 0x7f4a3806deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:34.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.173+0000 7f4a37400640 1 --2- 172.21.15.40:0/1605915874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a38144d50 0x7f4a38071470 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4a38072600 tx=0x7f4a2402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718ce0d2e9c6b2b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:34.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.173+0000 7f4a36a00640 1 -- 172.21.15.40:0/1605915874 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24035020 con 0x7f4a38144d50 2024-03-08T06:04:34.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.173+0000 7f4a36a00640 1 -- 172.21.15.40:0/1605915874 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a24031870 con 0x7f4a38144d50 2024-03-08T06:04:34.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.173+0000 7f4a36a00640 1 -- 172.21.15.40:0/1605915874 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24031c40 con 0x7f4a38144d50 2024-03-08T06:04:34.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.174+0000 7f4a3d800640 1 -- 172.21.15.40:0/1605915874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a38144d50 msgr2=0x7f4a38071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.174+0000 7f4a3d800640 1 --2- 172.21.15.40:0/1605915874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a38144d50 0x7f4a38071470 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4a38072600 tx=0x7f4a2402b910 comp rx=0 tx=0).stop 2024-03-08T06:04:34.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.174+0000 7f4a3d800640 1 -- 172.21.15.40:0/1605915874 shutdown_connections 2024-03-08T06:04:34.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.174+0000 7f4a3d800640 1 --2- 172.21.15.40:0/1605915874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a38144d50 0x7f4a38071470 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.174+0000 7f4a3d800640 1 --2- 172.21.15.40:0/1605915874 >> v2:172.21.15.53:3300/0 conn(0x7f4a381445d0 0x7f4a3806deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.174+0000 7f4a3d800640 1 -- 172.21.15.40:0/1605915874 >> 172.21.15.40:0/1605915874 conn(0x7f4a380b1600 msgr2=0x7f4a380b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:34.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.174+0000 7f4a3d800640 1 -- 172.21.15.40:0/1605915874 shutdown_connections 2024-03-08T06:04:34.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.174+0000 7f4a3d800640 1 -- 172.21.15.40:0/1605915874 wait complete. 2024-03-08T06:04:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.176+0000 7f4a3d800640 1 Processor -- start 2024-03-08T06:04:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.176+0000 7f4a3d800640 1 -- start start 2024-03-08T06:04:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.177+0000 7f4a3d800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a380719b0 0x7f4a3815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.177+0000 7f4a3d800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a381445d0 0x7f4a3813cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.177+0000 7f4a3d800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a38144d50 0x7f4a380b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.177+0000 7f4a3d800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380cc340 con 0x7f4a38144d50 2024-03-08T06:04:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.177+0000 7f4a3d800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380cc060 con 0x7f4a381445d0 2024-03-08T06:04:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.177+0000 7f4a3d800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a381449b0 con 0x7f4a380719b0 2024-03-08T06:04:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.177+0000 7f4a3ce00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a38144d50 0x7f4a380b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.177+0000 7f4a3ce00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a38144d50 0x7f4a380b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59086/0 (socket says 172.21.15.40:59086) 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.177+0000 7f4a3ce00640 1 -- 172.21.15.40:0/3894276645 learned_addr learned my addr 172.21.15.40:0/3894276645 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.178+0000 7f4a37400640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a381445d0 0x7f4a3813cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.178+0000 7f4a37e00640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a380719b0 0x7f4a3815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.178+0000 7f4a3ce00640 1 -- 172.21.15.40:0/3894276645 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a380719b0 msgr2=0x7f4a3815c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.178+0000 7f4a3ce00640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a380719b0 0x7f4a3815c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.178+0000 7f4a3ce00640 1 -- 172.21.15.40:0/3894276645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a381445d0 msgr2=0x7f4a3813cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.178+0000 7f4a3ce00640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a381445d0 0x7f4a3813cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.178+0000 7f4a3ce00640 1 -- 172.21.15.40:0/3894276645 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3813d1b0 con 0x7f4a38144d50 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.178+0000 7f4a3ce00640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a38144d50 0x7f4a380b3790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000b970 tx=0x7f4a3000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.178+0000 7f4a2fe00640 1 -- 172.21.15.40:0/3894276645 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30019070 con 0x7f4a38144d50 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.179+0000 7f4a2fe00640 1 -- 172.21.15.40:0/3894276645 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000cb40 con 0x7f4a38144d50 2024-03-08T06:04:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.179+0000 7f4a2fe00640 1 -- 172.21.15.40:0/3894276645 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30014440 con 0x7f4a38144d50 2024-03-08T06:04:34.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.179+0000 7f4a3d800640 1 -- 172.21.15.40:0/3894276645 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3815c680 con 0x7f4a38144d50 2024-03-08T06:04:34.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.179+0000 7f4a3d800640 1 -- 172.21.15.40:0/3894276645 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3815cc40 con 0x7f4a38144d50 2024-03-08T06:04:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.180+0000 7f4a3d800640 1 -- 172.21.15.40:0/3894276645 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a380ce0f0 con 0x7f4a38144d50 2024-03-08T06:04:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.181+0000 7f4a2fe00640 1 -- 172.21.15.40:0/3894276645 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a3001c070 con 0x7f4a38144d50 2024-03-08T06:04:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.182+0000 7f4a2fe00640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4a1806b4d0 0x7f4a1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.182+0000 7f4a2fe00640 1 -- 172.21.15.40:0/3894276645 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a30089e30 con 0x7f4a38144d50 2024-03-08T06:04:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.182+0000 7f4a37e00640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4a1806b4d0 0x7f4a1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.183+0000 7f4a37e00640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4a1806b4d0 0x7f4a1806d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800a950 tx=0x7f4a28008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:34.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.186+0000 7f4a2fe00640 1 -- 172.21.15.40:0/3894276645 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a380ce0f0 con 0x7f4a38144d50 2024-03-08T06:04:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.386+0000 7f4a3d800640 1 -- 172.21.15.40:0/3894276645 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a380af350 con 0x7f4a38144d50 2024-03-08T06:04:34.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.389+0000 7f4a2fe00640 1 -- 172.21.15.40:0/3894276645 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f4a3813d1b0 con 0x7f4a38144d50 2024-03-08T06:04:34.391 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:34.391 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:34.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.391+0000 7f4a3d800640 1 -- 172.21.15.40:0/3894276645 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4a1806b4d0 msgr2=0x7f4a1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.392+0000 7f4a3d800640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4a1806b4d0 0x7f4a1806d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800a950 tx=0x7f4a28008040 comp rx=0 tx=0).stop 2024-03-08T06:04:34.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.392+0000 7f4a3d800640 1 -- 172.21.15.40:0/3894276645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a38144d50 msgr2=0x7f4a380b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.392+0000 7f4a3d800640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a38144d50 0x7f4a380b3790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000b970 tx=0x7f4a3000be30 comp rx=0 tx=0).stop 2024-03-08T06:04:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.392+0000 7f4a3d800640 1 -- 172.21.15.40:0/3894276645 shutdown_connections 2024-03-08T06:04:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.392+0000 7f4a3d800640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f4a1806b4d0 0x7f4a1806d990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.392+0000 7f4a3d800640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a38144d50 0x7f4a380b3790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.392+0000 7f4a3d800640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a381445d0 0x7f4a3813cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.392+0000 7f4a3d800640 1 --2- 172.21.15.40:0/3894276645 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a380719b0 0x7f4a3815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.392+0000 7f4a3d800640 1 -- 172.21.15.40:0/3894276645 >> 172.21.15.40:0/3894276645 conn(0x7f4a380b1600 msgr2=0x7f4a380aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.392+0000 7f4a3d800640 1 -- 172.21.15.40:0/3894276645 shutdown_connections 2024-03-08T06:04:34.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.392+0000 7f4a3d800640 1 -- 172.21.15.40:0/3894276645 wait complete. 2024-03-08T06:04:34.412 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:34.413 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.639+0000 7ff621600640 1 Processor -- start 2024-03-08T06:04:34.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.640+0000 7ff621600640 1 -- start start 2024-03-08T06:04:34.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.641+0000 7ff621600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff61c071bc0 0x7ff61c06e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.641+0000 7ff621600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c0b3120 con 0x7ff61c1623a0 2024-03-08T06:04:34.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.641+0000 7ff621600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c0b3400 con 0x7ff61c071fa0 2024-03-08T06:04:34.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.641+0000 7ff621600640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c0b36e0 con 0x7ff61c071bc0 2024-03-08T06:04:34.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.642+0000 7ff61b400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff61c1623a0 0x7ff61c0ce1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56748/0 (socket says 172.21.15.40:56748) 2024-03-08T06:04:34.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.642+0000 7ff61b400640 1 -- 172.21.15.40:0/4037603630 learned_addr learned my addr 172.21.15.40:0/4037603630 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.642+0000 7ff61aa00640 1 --2- 172.21.15.40:0/4037603630 >> v2:172.21.15.53:3300/0 conn(0x7ff61c071bc0 0x7ff61c06e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.642+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380865696 0 0) 0x7ff61c0b3120 con 0x7ff61c1623a0 2024-03-08T06:04:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.642+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff600003520 con 0x7ff61c1623a0 2024-03-08T06:04:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 852741070 0 0) 0x7ff600003520 con 0x7ff61c1623a0 2024-03-08T06:04:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 >> v2:172.21.15.53:3300/0 conn(0x7ff61c071bc0 msgr2=0x7ff61c06e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 --2- 172.21.15.40:0/4037603630 >> v2:172.21.15.53:3300/0 conn(0x7ff61c071bc0 0x7ff61c06e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 >> v1:172.21.15.53:6789/0 conn(0x7ff61c071fa0 legacy=0x7ff61c161c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c0b39c0 con 0x7ff61c1623a0 2024-03-08T06:04:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7ff6140019f0 con 0x7ff61c1623a0 2024-03-08T06:04:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 >> v1:172.21.15.40:6789/0 conn(0x7ff61c1623a0 legacy=0x7ff61c0ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6000039b0 0x7ff600003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 0x7ff600004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff600005080 0x7ff60000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0b36e0 con 0x7ff6000043b0 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60000bfe0 con 0x7ff600005080 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60000c1e0 con 0x7ff6000039b0 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.643+0000 7ff613e00640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 0x7ff600004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.644+0000 7ff61b400640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff600005080 0x7ff60000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.644+0000 7ff61aa00640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6000039b0 0x7ff600003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.644+0000 7ff613e00640 1 -- 172.21.15.40:0/4037603630 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6000039b0 msgr2=0x7ff600003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.644+0000 7ff613e00640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6000039b0 0x7ff600003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.644+0000 7ff613e00640 1 -- 172.21.15.40:0/4037603630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff600005080 msgr2=0x7ff60000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.644+0000 7ff613e00640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff600005080 0x7ff60000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.644+0000 7ff613e00640 1 -- 172.21.15.40:0/4037603630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff600003520 con 0x7ff6000043b0 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.644+0000 7ff61b400640 1 -- 172.21.15.40:0/4037603630 reap_dead start 2024-03-08T06:04:34.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.644+0000 7ff61aa00640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6000039b0 0x7ff600003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:34.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.645+0000 7ff613e00640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 0x7ff600004880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff604001750 tx=0x7ff600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6850aac171b26 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:34.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.645+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604001370 con 0x7ff6000043b0 2024-03-08T06:04:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.646+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6040052d0 con 0x7ff6000043b0 2024-03-08T06:04:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.646+0000 7ff61a000640 1 -- 172.21.15.40:0/4037603630 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604003830 con 0x7ff6000043b0 2024-03-08T06:04:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.646+0000 7ff621600640 1 -- 172.21.15.40:0/4037603630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 msgr2=0x7ff600004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.646+0000 7ff621600640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 0x7ff600004880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff604001750 tx=0x7ff600005830 comp rx=0 tx=0).stop 2024-03-08T06:04:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.646+0000 7ff621600640 1 -- 172.21.15.40:0/4037603630 shutdown_connections 2024-03-08T06:04:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.646+0000 7ff621600640 1 --2- 172.21.15.40:0/4037603630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 0x7ff600004880 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.646+0000 7ff621600640 1 -- 172.21.15.40:0/4037603630 >> 172.21.15.40:0/4037603630 conn(0x7ff61c06ae30 msgr2=0x7ff61c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.646+0000 7ff621600640 1 -- 172.21.15.40:0/4037603630 shutdown_connections 2024-03-08T06:04:34.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.646+0000 7ff621600640 1 -- 172.21.15.40:0/4037603630 wait complete. 2024-03-08T06:04:34.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.648+0000 7ff621600640 1 Processor -- start 2024-03-08T06:04:34.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.648+0000 7ff621600640 1 -- start start 2024-03-08T06:04:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.649+0000 7ff621600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 0x7ff61c163cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.649+0000 7ff621600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff61c159940 0x7ff61c153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.649+0000 7ff621600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff61c1488d0 0x7ff61c148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.649+0000 7ff621600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0b2e30 con 0x7ff6000043b0 2024-03-08T06:04:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.649+0000 7ff621600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0b2cb0 con 0x7ff61c159940 2024-03-08T06:04:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.649+0000 7ff621600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c161c90 con 0x7ff61c1488d0 2024-03-08T06:04:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.649+0000 7ff61aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 0x7ff61c163cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.649+0000 7ff61aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 0x7ff61c163cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59106/0 (socket says 172.21.15.40:59106) 2024-03-08T06:04:34.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.649+0000 7ff61aa00640 1 -- 172.21.15.40:0/3548288463 learned_addr learned my addr 172.21.15.40:0/3548288463 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.650+0000 7ff613e00640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff61c159940 0x7ff61c153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.650+0000 7ff61b400640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff61c1488d0 0x7ff61c148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.650+0000 7ff61aa00640 1 -- 172.21.15.40:0/3548288463 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff61c1488d0 msgr2=0x7ff61c148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.650+0000 7ff61aa00640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff61c1488d0 0x7ff61c148d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.650+0000 7ff61aa00640 1 -- 172.21.15.40:0/3548288463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff61c159940 msgr2=0x7ff61c153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.650+0000 7ff61aa00640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff61c159940 0x7ff61c153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.650+0000 7ff61aa00640 1 -- 172.21.15.40:0/3548288463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c1540a0 con 0x7ff6000043b0 2024-03-08T06:04:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.650+0000 7ff613e00640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff61c159940 0x7ff61c153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.650+0000 7ff61aa00640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 0x7ff61c163cd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0107c0 tx=0x7ff60c010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.651+0000 7ff612a00640 1 -- 172.21.15.40:0/3548288463 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c00f040 con 0x7ff6000043b0 2024-03-08T06:04:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.651+0000 7ff612a00640 1 -- 172.21.15.40:0/3548288463 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c00eb60 con 0x7ff6000043b0 2024-03-08T06:04:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.651+0000 7ff612a00640 1 -- 172.21.15.40:0/3548288463 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c010e70 con 0x7ff6000043b0 2024-03-08T06:04:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.651+0000 7ff621600640 1 -- 172.21.15.40:0/3548288463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c154380 con 0x7ff6000043b0 2024-03-08T06:04:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.651+0000 7ff621600640 1 -- 172.21.15.40:0/3548288463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c154660 con 0x7ff6000043b0 2024-03-08T06:04:34.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.652+0000 7ff621600640 1 -- 172.21.15.40:0/3548288463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c067cb0 con 0x7ff6000043b0 2024-03-08T06:04:34.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.653+0000 7ff612a00640 1 -- 172.21.15.40:0/3548288463 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff60c01c400 con 0x7ff6000043b0 2024-03-08T06:04:34.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.654+0000 7ff612a00640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff5f806b500 0x7ff5f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:34.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.654+0000 7ff612a00640 1 -- 172.21.15.40:0/3548288463 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff60c08a8a0 con 0x7ff6000043b0 2024-03-08T06:04:34.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.654+0000 7ff613e00640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff5f806b500 0x7ff5f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:34.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.655+0000 7ff613e00640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff5f806b500 0x7ff5f806d9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff604033000 tx=0x7ff604009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:34.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.658+0000 7ff612a00640 1 -- 172.21.15.40:0/3548288463 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff60c018030 con 0x7ff6000043b0 2024-03-08T06:04:34.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.859+0000 7ff621600640 1 -- 172.21.15.40:0/3548288463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff61c15d080 con 0x7ff6000043b0 2024-03-08T06:04:34.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.860+0000 7ff612a00640 1 -- 172.21.15.40:0/3548288463 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff60c058fa0 con 0x7ff6000043b0 2024-03-08T06:04:34.861 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.862+0000 7ff621600640 1 -- 172.21.15.40:0/3548288463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff5f806b500 msgr2=0x7ff5f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.862+0000 7ff621600640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff5f806b500 0x7ff5f806d9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff604033000 tx=0x7ff604009030 comp rx=0 tx=0).stop 2024-03-08T06:04:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.862+0000 7ff621600640 1 -- 172.21.15.40:0/3548288463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 msgr2=0x7ff61c163cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.863+0000 7ff621600640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 0x7ff61c163cd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0107c0 tx=0x7ff60c010c80 comp rx=0 tx=0).stop 2024-03-08T06:04:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.863+0000 7ff621600640 1 -- 172.21.15.40:0/3548288463 shutdown_connections 2024-03-08T06:04:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.863+0000 7ff621600640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff5f806b500 0x7ff5f806d9c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.863+0000 7ff621600640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff61c1488d0 0x7ff61c148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.863+0000 7ff621600640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff61c159940 0x7ff61c153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.863+0000 7ff621600640 1 --2- 172.21.15.40:0/3548288463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6000043b0 0x7ff61c163cd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:34.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.863+0000 7ff621600640 1 -- 172.21.15.40:0/3548288463 >> 172.21.15.40:0/3548288463 conn(0x7ff61c06ae30 msgr2=0x7ff61c06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:34.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.863+0000 7ff621600640 1 -- 172.21.15.40:0/3548288463 shutdown_connections 2024-03-08T06:04:34.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:34.863+0000 7ff621600640 1 -- 172.21.15.40:0/3548288463 wait complete. 2024-03-08T06:04:34.881 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.093+0000 7fd7cf000640 1 Processor -- start 2024-03-08T06:04:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.093+0000 7fd7cf000640 1 -- start start 2024-03-08T06:04:35.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.095+0000 7fd7cf000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7c81455b0 0x7fd7c815c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.095+0000 7fd7cf000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c80cd5a0 con 0x7fd7c81604f0 2024-03-08T06:04:35.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.095+0000 7fd7cf000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c80ccfe0 con 0x7fd7c8071950 2024-03-08T06:04:35.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.095+0000 7fd7cf000640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c80ccda0 con 0x7fd7c81455b0 2024-03-08T06:04:35.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.095+0000 7fd7c7400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd7c81604f0 0x7fd7c80cc690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56750/0 (socket says 172.21.15.40:56750) 2024-03-08T06:04:35.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.095+0000 7fd7c7400640 1 -- 172.21.15.40:0/2855530026 learned_addr learned my addr 172.21.15.40:0/2855530026 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:35.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.095+0000 7fd7c7e00640 1 --2- 172.21.15.40:0/2855530026 >> v2:172.21.15.40:3300/0 conn(0x7fd7c81455b0 0x7fd7c815c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.096+0000 7fd7c7e00640 1 -- 172.21.15.40:0/2855530026 >> v1:172.21.15.53:6789/0 conn(0x7fd7c8071950 legacy=0x7fd7c80ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:35.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.096+0000 7fd7c7e00640 1 -- 172.21.15.40:0/2855530026 >> v1:172.21.15.40:6789/0 conn(0x7fd7c81604f0 legacy=0x7fd7c80cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.096+0000 7fd7c7e00640 1 -- 172.21.15.40:0/2855530026 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c80cd2c0 con 0x7fd7c81455b0 2024-03-08T06:04:35.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.097+0000 7fd7c7e00640 1 --2- 172.21.15.40:0/2855530026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c81455b0 0x7fd7c815c1d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009d90 tx=0x7fd7b802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6101823ed0174d50 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:35.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.097+0000 7fd7c6a00640 1 -- 172.21.15.40:0/2855530026 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8039070 con 0x7fd7c81455b0 2024-03-08T06:04:35.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.097+0000 7fd7c6a00640 1 -- 172.21.15.40:0/2855530026 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b802cc60 con 0x7fd7c81455b0 2024-03-08T06:04:35.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.097+0000 7fd7c6a00640 1 -- 172.21.15.40:0/2855530026 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8034660 con 0x7fd7c81455b0 2024-03-08T06:04:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.098+0000 7fd7cf000640 1 -- 172.21.15.40:0/2855530026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c81455b0 msgr2=0x7fd7c815c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.098+0000 7fd7cf000640 1 --2- 172.21.15.40:0/2855530026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c81455b0 0x7fd7c815c1d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009d90 tx=0x7fd7b802beb0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.098+0000 7fd7cf000640 1 -- 172.21.15.40:0/2855530026 shutdown_connections 2024-03-08T06:04:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.098+0000 7fd7cf000640 1 --2- 172.21.15.40:0/2855530026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c81455b0 0x7fd7c815c1d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.098+0000 7fd7cf000640 1 -- 172.21.15.40:0/2855530026 >> 172.21.15.40:0/2855530026 conn(0x7fd7c806ae30 msgr2=0x7fd7c806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:35.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.098+0000 7fd7cf000640 1 -- 172.21.15.40:0/2855530026 shutdown_connections 2024-03-08T06:04:35.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.098+0000 7fd7cf000640 1 -- 172.21.15.40:0/2855530026 wait complete. 2024-03-08T06:04:35.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.100+0000 7fd7cf000640 1 Processor -- start 2024-03-08T06:04:35.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.100+0000 7fd7cf000640 1 -- start start 2024-03-08T06:04:35.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.101+0000 7fd7cf000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd7c8071950 0x7fd7c81f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.101+0000 7fd7cf000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c81455b0 0x7fd7c81f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.101+0000 7fd7cf000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7c81604f0 0x7fd7c813e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.101+0000 7fd7cf000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c815c710 con 0x7fd7c81455b0 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.101+0000 7fd7cf000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c80cdc20 con 0x7fd7c81604f0 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.101+0000 7fd7cf000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c8145990 con 0x7fd7c8071950 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.101+0000 7fd7c7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c81455b0 0x7fd7c81f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.101+0000 7fd7c7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c81455b0 0x7fd7c81f0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59136/0 (socket says 172.21.15.40:59136) 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.101+0000 7fd7c7400640 1 -- 172.21.15.40:0/1621021801 learned_addr learned my addr 172.21.15.40:0/1621021801 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.102+0000 7fd7c7e00640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd7c8071950 0x7fd7c81f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.102+0000 7fd7cd000640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7c81604f0 0x7fd7c813e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.102+0000 7fd7c7400640 1 -- 172.21.15.40:0/1621021801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd7c8071950 msgr2=0x7fd7c81f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.102+0000 7fd7c7400640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd7c8071950 0x7fd7c81f3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.102+0000 7fd7c7400640 1 -- 172.21.15.40:0/1621021801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7c81604f0 msgr2=0x7fd7c813e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.102+0000 7fd7c7400640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7c81604f0 0x7fd7c813e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.102+0000 7fd7c7400640 1 -- 172.21.15.40:0/1621021801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c80b3ab0 con 0x7fd7c81455b0 2024-03-08T06:04:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.102+0000 7fd7cd000640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7c81604f0 0x7fd7c813e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.102+0000 7fd7c7400640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c81455b0 0x7fd7c81f0ed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c470 tx=0x7fd7b400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.102+0000 7fd7bfe00640 1 -- 172.21.15.40:0/1621021801 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4017070 con 0x7fd7c81455b0 2024-03-08T06:04:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.102+0000 7fd7bfe00640 1 -- 172.21.15.40:0/1621021801 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b4005ce0 con 0x7fd7c81455b0 2024-03-08T06:04:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.103+0000 7fd7bfe00640 1 -- 172.21.15.40:0/1621021801 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40072d0 con 0x7fd7c81455b0 2024-03-08T06:04:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.103+0000 7fd7cf000640 1 -- 172.21.15.40:0/1621021801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c80b3d90 con 0x7fd7c81455b0 2024-03-08T06:04:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.103+0000 7fd7cf000640 1 -- 172.21.15.40:0/1621021801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c80b4350 con 0x7fd7c81455b0 2024-03-08T06:04:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.104+0000 7fd7cf000640 1 -- 172.21.15.40:0/1621021801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c8067cb0 con 0x7fd7c81455b0 2024-03-08T06:04:35.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.105+0000 7fd7bfe00640 1 -- 172.21.15.40:0/1621021801 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7b401a070 con 0x7fd7c81455b0 2024-03-08T06:04:35.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.106+0000 7fd7bfe00640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd7a806b4d0 0x7fd7a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.106+0000 7fd7bfe00640 1 -- 172.21.15.40:0/1621021801 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7b4089400 con 0x7fd7c81455b0 2024-03-08T06:04:35.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.106+0000 7fd7c7e00640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd7a806b4d0 0x7fd7a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.107+0000 7fd7c7e00640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd7a806b4d0 0x7fd7a806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8036000 tx=0x7fd7b8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:35.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.110+0000 7fd7bfe00640 1 -- 172.21.15.40:0/1621021801 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7b4057c20 con 0x7fd7c81455b0 2024-03-08T06:04:35.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.312+0000 7fd7cf000640 1 -- 172.21.15.40:0/1621021801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7c80729c0 con 0x7fd7c81455b0 2024-03-08T06:04:35.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.312+0000 7fd7bfe00640 1 -- 172.21.15.40:0/1621021801 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd7b405bcc0 con 0x7fd7c81455b0 2024-03-08T06:04:35.313 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.315+0000 7fd7cf000640 1 -- 172.21.15.40:0/1621021801 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd7a806b4d0 msgr2=0x7fd7a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.315+0000 7fd7cf000640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd7a806b4d0 0x7fd7a806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8036000 tx=0x7fd7b8030000 comp rx=0 tx=0).stop 2024-03-08T06:04:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.315+0000 7fd7cf000640 1 -- 172.21.15.40:0/1621021801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c81455b0 msgr2=0x7fd7c81f0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.315+0000 7fd7cf000640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c81455b0 0x7fd7c81f0ed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c470 tx=0x7fd7b400c930 comp rx=0 tx=0).stop 2024-03-08T06:04:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.315+0000 7fd7cf000640 1 -- 172.21.15.40:0/1621021801 shutdown_connections 2024-03-08T06:04:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.315+0000 7fd7cf000640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd7a806b4d0 0x7fd7a806d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.315+0000 7fd7cf000640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7c81604f0 0x7fd7c813e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.315+0000 7fd7cf000640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c81455b0 0x7fd7c81f0ed0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.315+0000 7fd7cf000640 1 --2- 172.21.15.40:0/1621021801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd7c8071950 0x7fd7c81f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.315+0000 7fd7cf000640 1 -- 172.21.15.40:0/1621021801 >> 172.21.15.40:0/1621021801 conn(0x7fd7c806ae30 msgr2=0x7fd7c806dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.316+0000 7fd7cf000640 1 -- 172.21.15.40:0/1621021801 shutdown_connections 2024-03-08T06:04:35.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.316+0000 7fd7cf000640 1 -- 172.21.15.40:0/1621021801 wait complete. 2024-03-08T06:04:35.337 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:35.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.561+0000 7fd4ebe00640 1 Processor -- start 2024-03-08T06:04:35.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.562+0000 7fd4ebe00640 1 -- start start 2024-03-08T06:04:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.563+0000 7fd4ebe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4ec15de00 0x7fd4ec0b9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.563+0000 7fd4ebe00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd4ec0cefd0 0x7fd4ec0b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.563+0000 7fd4ebe00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ec0d1fa0 con 0x7fd4ec0d3040 2024-03-08T06:04:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.563+0000 7fd4ebe00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec0d2280 con 0x7fd4ec0cefd0 2024-03-08T06:04:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.563+0000 7fd4ebe00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0d2560 con 0x7fd4ec15de00 2024-03-08T06:04:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.563+0000 7fd4ea000640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd4ec0d3040 0x7fd4ec0ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56756/0 (socket says 172.21.15.40:56756) 2024-03-08T06:04:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.564+0000 7fd4ea000640 1 -- 172.21.15.40:0/3615558285 learned_addr learned my addr 172.21.15.40:0/3615558285 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.564+0000 7fd4eb400640 1 --2- 172.21.15.40:0/3615558285 >> v2:172.21.15.53:3300/0 conn(0x7fd4ec0cefd0 0x7fd4ec0b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.564+0000 7fd4eaa00640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4ec15de00 0x7fd4ec0b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.564+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042462493 0 0) 0x7fd4ec0d1fa0 con 0x7fd4ec0d3040 2024-03-08T06:04:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.564+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4cc003520 con 0x7fd4ec0d3040 2024-03-08T06:04:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.564+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3357380418 0 0) 0x7fd4cc003520 con 0x7fd4ec0d3040 2024-03-08T06:04:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.564+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4ec15de00 msgr2=0x7fd4ec0b9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.564+0000 7fd4e9600640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4ec15de00 0x7fd4ec0b9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.564+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 >> v2:172.21.15.53:3300/0 conn(0x7fd4ec0cefd0 msgr2=0x7fd4ec0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4e9600640 1 --2- 172.21.15.40:0/3615558285 >> v2:172.21.15.53:3300/0 conn(0x7fd4ec0cefd0 0x7fd4ec0b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec0d2840 con 0x7fd4ec0d3040 2024-03-08T06:04:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fd4d40019f0 con 0x7fd4ec0d3040 2024-03-08T06:04:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 >> v1:172.21.15.40:6789/0 conn(0x7fd4ec0d3040 legacy=0x7fd4ec0ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4e9600640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4ec15de00 0x7fd4cc003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4e9600640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4cc003ff0 0x7fd4cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4e9600640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4cc004d40 0x7fd4cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0d2280 con 0x7fd4cc004d40 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0d2560 con 0x7fd4cc003ff0 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc00bac0 con 0x7fd4ec15de00 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4eb400640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4cc004d40 0x7fd4cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4eaa00640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4ec15de00 0x7fd4cc003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.565+0000 7fd4ea000640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4cc003ff0 0x7fd4cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.566+0000 7fd4eb400640 1 -- 172.21.15.40:0/3615558285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4ec15de00 msgr2=0x7fd4cc003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.566+0000 7fd4eb400640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4ec15de00 0x7fd4cc003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.566+0000 7fd4eb400640 1 -- 172.21.15.40:0/3615558285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4cc003ff0 msgr2=0x7fd4cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.566+0000 7fd4eb400640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4cc003ff0 0x7fd4cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.566+0000 7fd4eb400640 1 -- 172.21.15.40:0/3615558285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc003520 con 0x7fd4cc004d40 2024-03-08T06:04:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.567+0000 7fd4eb400640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4cc004d40 0x7fd4cc0051c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009a90 tx=0x7fd4cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e5d5d1a6dc6755 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.567+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4039070 con 0x7fd4cc004d40 2024-03-08T06:04:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.567+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e402c6b0 con 0x7fd4cc004d40 2024-03-08T06:04:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.567+0000 7fd4e9600640 1 -- 172.21.15.40:0/3615558285 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4009690 con 0x7fd4cc004d40 2024-03-08T06:04:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.568+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3615558285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4cc004d40 msgr2=0x7fd4cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.568+0000 7fd4ebe00640 1 --2- 172.21.15.40:0/3615558285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4cc004d40 0x7fd4cc0051c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009a90 tx=0x7fd4cc005830 comp rx=0 tx=0).stop 2024-03-08T06:04:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.568+0000 7fd4eb400640 1 -- 172.21.15.40:0/3615558285 reap_dead start 2024-03-08T06:04:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.568+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3615558285 shutdown_connections 2024-03-08T06:04:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.568+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3615558285 >> 172.21.15.40:0/3615558285 conn(0x7fd4ec066fc0 msgr2=0x7fd4ec0b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.568+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3615558285 shutdown_connections 2024-03-08T06:04:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.568+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3615558285 wait complete. 2024-03-08T06:04:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.570+0000 7fd4ebe00640 1 Processor -- start 2024-03-08T06:04:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.570+0000 7fd4ebe00640 1 -- start start 2024-03-08T06:04:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.571+0000 7fd4ebe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4cc004d40 0x7fd4ec1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.571+0000 7fd4ebe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ec15ba00 0x7fd4ec153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.571+0000 7fd4ebe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec15be80 0x7fd4ec154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.571+0000 7fd4ebe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0d1be0 con 0x7fd4ec15be80 2024-03-08T06:04:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.571+0000 7fd4ebe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec15e1e0 con 0x7fd4ec15ba00 2024-03-08T06:04:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.571+0000 7fd4ebe00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0d1d60 con 0x7fd4cc004d40 2024-03-08T06:04:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.571+0000 7fd4eb400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec15be80 0x7fd4ec154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.571+0000 7fd4eb400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec15be80 0x7fd4ec154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59162/0 (socket says 172.21.15.40:59162) 2024-03-08T06:04:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.571+0000 7fd4eb400640 1 -- 172.21.15.40:0/3356752129 learned_addr learned my addr 172.21.15.40:0/3356752129 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4ea000640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ec15ba00 0x7fd4ec153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4eb400640 1 -- 172.21.15.40:0/3356752129 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4cc004d40 msgr2=0x7fd4ec1558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4eaa00640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4cc004d40 0x7fd4ec1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4eb400640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4cc004d40 0x7fd4ec1558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4eb400640 1 -- 172.21.15.40:0/3356752129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ec15ba00 msgr2=0x7fd4ec153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4eb400640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ec15ba00 0x7fd4ec153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4eb400640 1 -- 172.21.15.40:0/3356752129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec154850 con 0x7fd4ec15be80 2024-03-08T06:04:35.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4ea000640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ec15ba00 0x7fd4ec153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:35.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4eb400640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec15be80 0x7fd4ec154180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009a90 tx=0x7fd4e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:35.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4e2a00640 1 -- 172.21.15.40:0/3356752129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4039070 con 0x7fd4ec15be80 2024-03-08T06:04:35.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4e2a00640 1 -- 172.21.15.40:0/3356752129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e4043070 con 0x7fd4ec15be80 2024-03-08T06:04:35.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.572+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3356752129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec154b30 con 0x7fd4ec15be80 2024-03-08T06:04:35.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.573+0000 7fd4e2a00640 1 -- 172.21.15.40:0/3356752129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4009690 con 0x7fd4ec15be80 2024-03-08T06:04:35.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.573+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3356752129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec1550f0 con 0x7fd4ec15be80 2024-03-08T06:04:35.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.574+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3356752129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ec075090 con 0x7fd4ec15be80 2024-03-08T06:04:35.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.575+0000 7fd4e2a00640 1 -- 172.21.15.40:0/3356752129 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4e4003850 con 0x7fd4ec15be80 2024-03-08T06:04:35.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.576+0000 7fd4e2a00640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd4c806b4d0 0x7fd4c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:35.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.576+0000 7fd4e2a00640 1 -- 172.21.15.40:0/3356752129 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4e40ab260 con 0x7fd4ec15be80 2024-03-08T06:04:35.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.576+0000 7fd4eaa00640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd4c806b4d0 0x7fd4c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:35.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.577+0000 7fd4eaa00640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd4c806b4d0 0x7fd4c806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009950 tx=0x7fd4dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:35.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.580+0000 7fd4e2a00640 1 -- 172.21.15.40:0/3356752129 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4e407a040 con 0x7fd4ec15be80 2024-03-08T06:04:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.780+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3356752129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4ec0b4300 con 0x7fd4ec15be80 2024-03-08T06:04:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.780+0000 7fd4e2a00640 1 -- 172.21.15.40:0/3356752129 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4e4079940 con 0x7fd4ec15be80 2024-03-08T06:04:35.781 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:35.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3356752129 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd4c806b4d0 msgr2=0x7fd4c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd4c806b4d0 0x7fd4c806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009950 tx=0x7fd4dc008040 comp rx=0 tx=0).stop 2024-03-08T06:04:35.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3356752129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec15be80 msgr2=0x7fd4ec154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:35.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec15be80 0x7fd4ec154180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009a90 tx=0x7fd4e4004730 comp rx=0 tx=0).stop 2024-03-08T06:04:35.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3356752129 shutdown_connections 2024-03-08T06:04:35.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd4c806b4d0 0x7fd4c806d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec15be80 0x7fd4ec154180 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ec15ba00 0x7fd4ec153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 --2- 172.21.15.40:0/3356752129 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4cc004d40 0x7fd4ec1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3356752129 >> 172.21.15.40:0/3356752129 conn(0x7fd4ec066fc0 msgr2=0x7fd4ec0b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3356752129 shutdown_connections 2024-03-08T06:04:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:35.783+0000 7fd4ebe00640 1 -- 172.21.15.40:0/3356752129 wait complete. 2024-03-08T06:04:35.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:36.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.017+0000 7f98f1a00640 1 Processor -- start 2024-03-08T06:04:36.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.018+0000 7f98f1a00640 1 -- start start 2024-03-08T06:04:36.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.019+0000 7f98f1a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98ec06d600 0x7f98ec144d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.019+0000 7f98f1a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98ec071e90 0x7f98ec1452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.019+0000 7f98f1a00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ec06da80 con 0x7f98ec071ab0 2024-03-08T06:04:36.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.019+0000 7f98f1a00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ec06ef40 con 0x7f98ec071e90 2024-03-08T06:04:36.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.019+0000 7f98f1a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec1457e0 con 0x7f98ec06d600 2024-03-08T06:04:36.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.020+0000 7f98eaa00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f98ec071ab0 0x7f98ec15cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56766/0 (socket says 172.21.15.40:56766) 2024-03-08T06:04:36.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.020+0000 7f98eaa00640 1 -- 172.21.15.40:0/3251544548 learned_addr learned my addr 172.21.15.40:0/3251544548 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:36.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.020+0000 7f98eb400640 1 --2- 172.21.15.40:0/3251544548 >> v2:172.21.15.40:3300/0 conn(0x7f98ec071e90 0x7f98ec1452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.020+0000 7f98ea000640 1 --2- 172.21.15.40:0/3251544548 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98ec06d600 0x7f98ec144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.020+0000 7f98e9600640 1 -- 172.21.15.40:0/3251544548 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065030640 0 0) 0x7f98ec06da80 con 0x7f98ec071ab0 2024-03-08T06:04:36.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.020+0000 7f98e9600640 1 -- 172.21.15.40:0/3251544548 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98d0003520 con 0x7f98ec071ab0 2024-03-08T06:04:36.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.020+0000 7f98eb400640 1 -- 172.21.15.40:0/3251544548 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98ec06d600 msgr2=0x7f98ec144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:36.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.020+0000 7f98eb400640 1 --2- 172.21.15.40:0/3251544548 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98ec06d600 0x7f98ec144d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.020+0000 7f98eb400640 1 -- 172.21.15.40:0/3251544548 >> v1:172.21.15.40:6789/0 conn(0x7f98ec071ab0 legacy=0x7f98ec15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.020+0000 7f98eb400640 1 -- 172.21.15.40:0/3251544548 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec06dd60 con 0x7f98ec071e90 2024-03-08T06:04:36.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.022+0000 7f98eb400640 1 --2- 172.21.15.40:0/3251544548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec071e90 0x7f98ec1452a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009d90 tx=0x7f98e402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb01a8d1e7e7375 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:36.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.022+0000 7f98e9600640 1 -- 172.21.15.40:0/3251544548 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4039070 con 0x7f98ec071e90 2024-03-08T06:04:36.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.022+0000 7f98e9600640 1 -- 172.21.15.40:0/3251544548 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e402cc50 con 0x7f98ec071e90 2024-03-08T06:04:36.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.022+0000 7f98e9600640 1 -- 172.21.15.40:0/3251544548 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4034440 con 0x7f98ec071e90 2024-03-08T06:04:36.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.022+0000 7f98f1a00640 1 -- 172.21.15.40:0/3251544548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec071e90 msgr2=0x7f98ec1452a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.022+0000 7f98f1a00640 1 --2- 172.21.15.40:0/3251544548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec071e90 0x7f98ec1452a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009d90 tx=0x7f98e402bea0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.023+0000 7f98f1a00640 1 -- 172.21.15.40:0/3251544548 shutdown_connections 2024-03-08T06:04:36.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.023+0000 7f98f1a00640 1 --2- 172.21.15.40:0/3251544548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec071e90 0x7f98ec1452a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.023+0000 7f98f1a00640 1 --2- 172.21.15.40:0/3251544548 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98ec06d600 0x7f98ec144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.023+0000 7f98f1a00640 1 -- 172.21.15.40:0/3251544548 >> 172.21.15.40:0/3251544548 conn(0x7f98ec06ae30 msgr2=0x7f98ec06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:36.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.023+0000 7f98f1a00640 1 -- 172.21.15.40:0/3251544548 shutdown_connections 2024-03-08T06:04:36.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.023+0000 7f98f1a00640 1 -- 172.21.15.40:0/3251544548 wait complete. 2024-03-08T06:04:36.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.024+0000 7f98f1a00640 1 Processor -- start 2024-03-08T06:04:36.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.025+0000 7f98f1a00640 1 -- start start 2024-03-08T06:04:36.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98f1a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98ec06d600 0x7f98ec15b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98f1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98ec071ab0 0x7f98ec140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98f1a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec071e90 0x7f98ec140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98f1a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec06e600 con 0x7f98ec071e90 2024-03-08T06:04:36.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98f1a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec06e320 con 0x7f98ec071ab0 2024-03-08T06:04:36.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98f1a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec15d3a0 con 0x7f98ec06d600 2024-03-08T06:04:36.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98eb400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec071e90 0x7f98ec140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98eb400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec071e90 0x7f98ec140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59184/0 (socket says 172.21.15.40:59184) 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98eb400640 1 -- 172.21.15.40:0/525208303 learned_addr learned my addr 172.21.15.40:0/525208303 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98eaa00640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98ec06d600 0x7f98ec15b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98ea000640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98ec071ab0 0x7f98ec140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98eb400640 1 -- 172.21.15.40:0/525208303 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98ec06d600 msgr2=0x7f98ec15b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98eb400640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98ec06d600 0x7f98ec15b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98eb400640 1 -- 172.21.15.40:0/525208303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98ec071ab0 msgr2=0x7f98ec140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98eb400640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98ec071ab0 0x7f98ec140290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98eb400640 1 -- 172.21.15.40:0/525208303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec13bf80 con 0x7f98ec071e90 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.026+0000 7f98ea000640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98ec071ab0 0x7f98ec140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.027+0000 7f98eb400640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec071e90 0x7f98ec140b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009d90 tx=0x7f98e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.027+0000 7f98e2a00640 1 -- 172.21.15.40:0/525208303 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4039050 con 0x7f98ec071e90 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.027+0000 7f98e2a00640 1 -- 172.21.15.40:0/525208303 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e4045070 con 0x7f98ec071e90 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.027+0000 7f98e2a00640 1 -- 172.21.15.40:0/525208303 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4009450 con 0x7f98ec071e90 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.027+0000 7f98f1a00640 1 -- 172.21.15.40:0/525208303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ec13e6d0 con 0x7f98ec071e90 2024-03-08T06:04:36.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.027+0000 7f98f1a00640 1 -- 172.21.15.40:0/525208303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ec15b8e0 con 0x7f98ec071e90 2024-03-08T06:04:36.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.028+0000 7f98f1a00640 1 -- 172.21.15.40:0/525208303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ec067cb0 con 0x7f98ec071e90 2024-03-08T06:04:36.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.029+0000 7f98e2a00640 1 -- 172.21.15.40:0/525208303 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98e403e610 con 0x7f98ec071e90 2024-03-08T06:04:36.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.030+0000 7f98e2a00640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98c806b4d0 0x7f98c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.030+0000 7f98e2a00640 1 -- 172.21.15.40:0/525208303 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98e40acc60 con 0x7f98ec071e90 2024-03-08T06:04:36.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.030+0000 7f98eaa00640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98c806b4d0 0x7f98c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.031+0000 7f98eaa00640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98c806b4d0 0x7f98c806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f98dc005880 tx=0x7f98dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:36.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.035+0000 7f98e2a00640 1 -- 172.21.15.40:0/525208303 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98e407ba40 con 0x7f98ec071e90 2024-03-08T06:04:36.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.233+0000 7f98f1a00640 1 -- 172.21.15.40:0/525208303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98ec0af620 con 0x7f98ec071e90 2024-03-08T06:04:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.234+0000 7f98e2a00640 1 -- 172.21.15.40:0/525208303 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f98e407b340 con 0x7f98ec071e90 2024-03-08T06:04:36.236 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:36.236 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:36.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 -- 172.21.15.40:0/525208303 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98c806b4d0 msgr2=0x7f98c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98c806b4d0 0x7f98c806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f98dc005880 tx=0x7f98dc009040 comp rx=0 tx=0).stop 2024-03-08T06:04:36.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 -- 172.21.15.40:0/525208303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec071e90 msgr2=0x7f98ec140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec071e90 0x7f98ec140b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009d90 tx=0x7f98e4004a40 comp rx=0 tx=0).stop 2024-03-08T06:04:36.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 -- 172.21.15.40:0/525208303 shutdown_connections 2024-03-08T06:04:36.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98c806b4d0 0x7f98c806d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98ec071e90 0x7f98ec140b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98ec071ab0 0x7f98ec140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 --2- 172.21.15.40:0/525208303 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98ec06d600 0x7f98ec15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 -- 172.21.15.40:0/525208303 >> 172.21.15.40:0/525208303 conn(0x7f98ec06ae30 msgr2=0x7f98ec15cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:36.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 -- 172.21.15.40:0/525208303 shutdown_connections 2024-03-08T06:04:36.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.237+0000 7f98f1a00640 1 -- 172.21.15.40:0/525208303 wait complete. 2024-03-08T06:04:36.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:36.256 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.469+0000 7fc54de00640 1 Processor -- start 2024-03-08T06:04:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.470+0000 7fc54de00640 1 -- start start 2024-03-08T06:04:36.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.471+0000 7fc54de00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc54815de60 0x7fc548161930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.471+0000 7fc54de00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc5480b9630 0x7fc5480ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.471+0000 7fc54de00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54815e240 con 0x7fc5480cee60 2024-03-08T06:04:36.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.471+0000 7fc54de00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5480cf2d0 con 0x7fc5480b9630 2024-03-08T06:04:36.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.471+0000 7fc54de00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc548162030 con 0x7fc54815de60 2024-03-08T06:04:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.471+0000 7fc53fe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc5480b9630 0x7fc5480ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.472+0000 7fc53fe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc5480b9630 0x7fc5480ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59190/0 (socket says 172.21.15.40:59190) 2024-03-08T06:04:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.472+0000 7fc53fe00640 1 -- 172.21.15.40:0/3850729527 learned_addr learned my addr 172.21.15.40:0/3850729527 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.472+0000 7fc54ca00640 1 --2- 172.21.15.40:0/3850729527 >> v2:172.21.15.53:3300/0 conn(0x7fc54815de60 0x7fc548161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.472+0000 7fc53fe00640 1 -- 172.21.15.40:0/3850729527 >> v2:172.21.15.53:3300/0 conn(0x7fc54815de60 msgr2=0x7fc548161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:36.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.472+0000 7fc53fe00640 1 --2- 172.21.15.40:0/3850729527 >> v2:172.21.15.53:3300/0 conn(0x7fc54815de60 0x7fc548161930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.472+0000 7fc53fe00640 1 -- 172.21.15.40:0/3850729527 >> v1:172.21.15.53:6789/0 conn(0x7fc5480cee60 legacy=0x7fc5480b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:36.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.472+0000 7fc53fe00640 1 -- 172.21.15.40:0/3850729527 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc548162310 con 0x7fc5480b9630 2024-03-08T06:04:36.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.473+0000 7fc53fe00640 1 --2- 172.21.15.40:0/3850729527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5480b9630 0x7fc5480ce920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc5480b6740 tx=0x7fc53002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c9dab49f46f526 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.474+0000 7fc547e00640 1 -- 172.21.15.40:0/3850729527 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530035020 con 0x7fc5480b9630 2024-03-08T06:04:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.474+0000 7fc547e00640 1 -- 172.21.15.40:0/3850729527 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc530031870 con 0x7fc5480b9630 2024-03-08T06:04:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.474+0000 7fc547e00640 1 -- 172.21.15.40:0/3850729527 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530031c40 con 0x7fc5480b9630 2024-03-08T06:04:36.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.474+0000 7fc54de00640 1 -- 172.21.15.40:0/3850729527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5480b9630 msgr2=0x7fc5480ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.474+0000 7fc54de00640 1 --2- 172.21.15.40:0/3850729527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5480b9630 0x7fc5480ce920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc5480b6740 tx=0x7fc53002b910 comp rx=0 tx=0).stop 2024-03-08T06:04:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.474+0000 7fc54de00640 1 -- 172.21.15.40:0/3850729527 shutdown_connections 2024-03-08T06:04:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.474+0000 7fc54de00640 1 --2- 172.21.15.40:0/3850729527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5480b9630 0x7fc5480ce920 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.474+0000 7fc54de00640 1 --2- 172.21.15.40:0/3850729527 >> v2:172.21.15.53:3300/0 conn(0x7fc54815de60 0x7fc548161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.474+0000 7fc54de00640 1 -- 172.21.15.40:0/3850729527 >> 172.21.15.40:0/3850729527 conn(0x7fc54806c500 msgr2=0x7fc54806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.474+0000 7fc54de00640 1 -- 172.21.15.40:0/3850729527 shutdown_connections 2024-03-08T06:04:36.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.475+0000 7fc54de00640 1 -- 172.21.15.40:0/3850729527 wait complete. 2024-03-08T06:04:36.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.476+0000 7fc54de00640 1 Processor -- start 2024-03-08T06:04:36.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.476+0000 7fc54de00640 1 -- start start 2024-03-08T06:04:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.477+0000 7fc54de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5480b9630 0x7fc548153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.477+0000 7fc54de00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc5480cee60 0x7fc5480c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.477+0000 7fc54de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc54815de60 0x7fc5480b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.477+0000 7fc54de00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480b9a90 con 0x7fc54815de60 2024-03-08T06:04:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.477+0000 7fc54de00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480d20c0 con 0x7fc5480b9630 2024-03-08T06:04:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.477+0000 7fc54de00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5480d35f0 con 0x7fc5480cee60 2024-03-08T06:04:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.478+0000 7fc54d400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc54815de60 0x7fc5480b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.478+0000 7fc54d400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc54815de60 0x7fc5480b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59204/0 (socket says 172.21.15.40:59204) 2024-03-08T06:04:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.478+0000 7fc54d400640 1 -- 172.21.15.40:0/3437440719 learned_addr learned my addr 172.21.15.40:0/3437440719 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:36.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.478+0000 7fc53fe00640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc5480cee60 0x7fc5480c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.478+0000 7fc54ca00640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5480b9630 0x7fc548153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.478+0000 7fc54d400640 1 -- 172.21.15.40:0/3437440719 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc5480cee60 msgr2=0x7fc5480c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.478+0000 7fc54d400640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc5480cee60 0x7fc5480c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.478+0000 7fc54d400640 1 -- 172.21.15.40:0/3437440719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5480b9630 msgr2=0x7fc548153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.478+0000 7fc54d400640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5480b9630 0x7fc548153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.478+0000 7fc54d400640 1 -- 172.21.15.40:0/3437440719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc548153e00 con 0x7fc54815de60 2024-03-08T06:04:36.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.479+0000 7fc54d400640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc54815de60 0x7fc5480b4fc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc54000c7b0 tx=0x7fc54000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:36.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.479+0000 7fc545600640 1 -- 172.21.15.40:0/3437440719 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540004b60 con 0x7fc54815de60 2024-03-08T06:04:36.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.479+0000 7fc545600640 1 -- 172.21.15.40:0/3437440719 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc540004d00 con 0x7fc54815de60 2024-03-08T06:04:36.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.479+0000 7fc545600640 1 -- 172.21.15.40:0/3437440719 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54000aa90 con 0x7fc54815de60 2024-03-08T06:04:36.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.479+0000 7fc54de00640 1 -- 172.21.15.40:0/3437440719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5481538b0 con 0x7fc54815de60 2024-03-08T06:04:36.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.479+0000 7fc54de00640 1 -- 172.21.15.40:0/3437440719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5480c8c50 con 0x7fc54815de60 2024-03-08T06:04:36.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.480+0000 7fc54de00640 1 -- 172.21.15.40:0/3437440719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc548075090 con 0x7fc54815de60 2024-03-08T06:04:36.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.481+0000 7fc545600640 1 -- 172.21.15.40:0/3437440719 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc54000ac30 con 0x7fc54815de60 2024-03-08T06:04:36.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.482+0000 7fc545600640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc52406b4d0 0x7fc52406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.482+0000 7fc545600640 1 -- 172.21.15.40:0/3437440719 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc54008a560 con 0x7fc54815de60 2024-03-08T06:04:36.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.482+0000 7fc54ca00640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc52406b4d0 0x7fc52406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.483+0000 7fc54ca00640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc52406b4d0 0x7fc52406d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc538006fd0 tx=0x7fc538008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:36.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.487+0000 7fc545600640 1 -- 172.21.15.40:0/3437440719 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc540020070 con 0x7fc54815de60 2024-03-08T06:04:36.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.684+0000 7fc54de00640 1 -- 172.21.15.40:0/3437440719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5480b7d80 con 0x7fc54815de60 2024-03-08T06:04:36.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.685+0000 7fc545600640 1 -- 172.21.15.40:0/3437440719 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc540059340 con 0x7fc54815de60 2024-03-08T06:04:36.687 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:36.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 -- 172.21.15.40:0/3437440719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc52406b4d0 msgr2=0x7fc52406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc52406b4d0 0x7fc52406d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc538006fd0 tx=0x7fc538008040 comp rx=0 tx=0).stop 2024-03-08T06:04:36.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 -- 172.21.15.40:0/3437440719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc54815de60 msgr2=0x7fc5480b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc54815de60 0x7fc5480b4fc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc54000c7b0 tx=0x7fc54000cc70 comp rx=0 tx=0).stop 2024-03-08T06:04:36.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 -- 172.21.15.40:0/3437440719 shutdown_connections 2024-03-08T06:04:36.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc52406b4d0 0x7fc52406d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc54815de60 0x7fc5480b4fc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc5480cee60 0x7fc5480c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 --2- 172.21.15.40:0/3437440719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5480b9630 0x7fc548153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 -- 172.21.15.40:0/3437440719 >> 172.21.15.40:0/3437440719 conn(0x7fc54806c500 msgr2=0x7fc54806c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:36.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 -- 172.21.15.40:0/3437440719 shutdown_connections 2024-03-08T06:04:36.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.688+0000 7fc54de00640 1 -- 172.21.15.40:0/3437440719 wait complete. 2024-03-08T06:04:36.709 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:36.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.930+0000 7f6049000640 1 Processor -- start 2024-03-08T06:04:36.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.930+0000 7f6049000640 1 -- start start 2024-03-08T06:04:36.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.932+0000 7f6049000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6044072060 0x7f604415e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.932+0000 7f6049000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f604415eb50 0x7f60440ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.932+0000 7f6049000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604415f390 con 0x7f60441584a0 2024-03-08T06:04:36.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.932+0000 7f6049000640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604406d820 con 0x7f604415eb50 2024-03-08T06:04:36.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.932+0000 7f6049000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604406db00 con 0x7f6044072060 2024-03-08T06:04:36.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.932+0000 7f6041e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f60441584a0 0x7f6044071950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42612/0 (socket says 172.21.15.40:42612) 2024-03-08T06:04:36.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.932+0000 7f6042800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f604415eb50 0x7f60440ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.932+0000 7f6041e00640 1 -- 172.21.15.40:0/3958277579 learned_addr learned my addr 172.21.15.40:0/3958277579 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:36.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.932+0000 7f6039400640 1 --2- 172.21.15.40:0/3958277579 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6044072060 0x7f604415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.933+0000 7f6042800640 1 -- 172.21.15.40:0/3958277579 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6044072060 msgr2=0x7f604415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:36.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.933+0000 7f6042800640 1 --2- 172.21.15.40:0/3958277579 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6044072060 0x7f604415e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.933+0000 7f6042800640 1 -- 172.21.15.40:0/3958277579 >> v1:172.21.15.40:6789/0 conn(0x7f60441584a0 legacy=0x7f6044071950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.933+0000 7f6041400640 1 -- 172.21.15.40:0/3958277579 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476097483 0 0) 0x7f604415f390 con 0x7f60441584a0 2024-03-08T06:04:36.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.933+0000 7f6042800640 1 -- 172.21.15.40:0/3958277579 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604406dde0 con 0x7f604415eb50 2024-03-08T06:04:36.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.934+0000 7f6042800640 1 --2- 172.21.15.40:0/3958277579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f604415eb50 0x7f60440ae1d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f603c009d90 tx=0x7f603c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f9a46ee14f038 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:36.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.934+0000 7f6041400640 1 -- 172.21.15.40:0/3958277579 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c039070 con 0x7f604415eb50 2024-03-08T06:04:36.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.934+0000 7f6041400640 1 -- 172.21.15.40:0/3958277579 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c02cbd0 con 0x7f604415eb50 2024-03-08T06:04:36.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.934+0000 7f6041400640 1 -- 172.21.15.40:0/3958277579 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c034440 con 0x7f604415eb50 2024-03-08T06:04:36.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.935+0000 7f6049000640 1 -- 172.21.15.40:0/3958277579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f604415eb50 msgr2=0x7f60440ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.935+0000 7f6049000640 1 --2- 172.21.15.40:0/3958277579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f604415eb50 0x7f60440ae1d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f603c009d90 tx=0x7f603c02be20 comp rx=0 tx=0).stop 2024-03-08T06:04:36.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.935+0000 7f6049000640 1 -- 172.21.15.40:0/3958277579 shutdown_connections 2024-03-08T06:04:36.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.935+0000 7f6049000640 1 --2- 172.21.15.40:0/3958277579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f604415eb50 0x7f60440ae1d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.935+0000 7f6049000640 1 --2- 172.21.15.40:0/3958277579 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6044072060 0x7f604415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.935+0000 7f6049000640 1 -- 172.21.15.40:0/3958277579 >> 172.21.15.40:0/3958277579 conn(0x7f604406ae30 msgr2=0x7f604406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:36.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.935+0000 7f6049000640 1 -- 172.21.15.40:0/3958277579 shutdown_connections 2024-03-08T06:04:36.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.935+0000 7f6049000640 1 -- 172.21.15.40:0/3958277579 wait complete. 2024-03-08T06:04:36.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.937+0000 7f6049000640 1 Processor -- start 2024-03-08T06:04:36.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.937+0000 7f6049000640 1 -- start start 2024-03-08T06:04:36.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.938+0000 7f6049000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6044072060 0x7f6044159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.938+0000 7f6049000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f60441584a0 0x7f604415b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.938+0000 7f6049000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604415eb50 0x7f60440af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.938+0000 7f6049000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604406e5e0 con 0x7f6044072060 2024-03-08T06:04:36.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.938+0000 7f6049000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604415efe0 con 0x7f604415eb50 2024-03-08T06:04:36.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.938+0000 7f6049000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604415a520 con 0x7f60441584a0 2024-03-08T06:04:36.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.938+0000 7f6041e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6044072060 0x7f6044159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.938+0000 7f6041e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6044072060 0x7f6044159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45208/0 (socket says 172.21.15.40:45208) 2024-03-08T06:04:36.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.938+0000 7f6041e00640 1 -- 172.21.15.40:0/800643133 learned_addr learned my addr 172.21.15.40:0/800643133 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:36.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f6039400640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f60441584a0 0x7f604415b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f6042800640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604415eb50 0x7f60440af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f6041e00640 1 -- 172.21.15.40:0/800643133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f60441584a0 msgr2=0x7f604415b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f6041e00640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f60441584a0 0x7f604415b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f6041e00640 1 -- 172.21.15.40:0/800643133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604415eb50 msgr2=0x7f60440af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:36.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f6041e00640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604415eb50 0x7f60440af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:36.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f6041e00640 1 -- 172.21.15.40:0/800643133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6044158f40 con 0x7f6044072060 2024-03-08T06:04:36.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f6039400640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f60441584a0 0x7f604415b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:36.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f6042800640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604415eb50 0x7f60440af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:36.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f6041e00640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6044072060 0x7f6044159fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f603400c460 tx=0x7f603400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:36.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f603aa00640 1 -- 172.21.15.40:0/800643133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034018070 con 0x7f6044072060 2024-03-08T06:04:36.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f603aa00640 1 -- 172.21.15.40:0/800643133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6034007500 con 0x7f6044072060 2024-03-08T06:04:36.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f603aa00640 1 -- 172.21.15.40:0/800643133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034004a20 con 0x7f6044072060 2024-03-08T06:04:36.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.939+0000 7f6049000640 1 -- 172.21.15.40:0/800643133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6044159220 con 0x7f6044072060 2024-03-08T06:04:36.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.940+0000 7f6049000640 1 -- 172.21.15.40:0/800643133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60441597e0 con 0x7f6044072060 2024-03-08T06:04:36.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.941+0000 7f6049000640 1 -- 172.21.15.40:0/800643133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6044067cb0 con 0x7f6044072060 2024-03-08T06:04:36.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.942+0000 7f603aa00640 1 -- 172.21.15.40:0/800643133 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6034004bc0 con 0x7f6044072060 2024-03-08T06:04:36.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.943+0000 7f603aa00640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f602006b4d0 0x7f602006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:36.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.943+0000 7f603aa00640 1 -- 172.21.15.40:0/800643133 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6034089620 con 0x7f6044072060 2024-03-08T06:04:36.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.943+0000 7f6039400640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f602006b4d0 0x7f602006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:36.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.944+0000 7f6039400640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f602006b4d0 0x7f602006d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f602c006fd0 tx=0x7f602c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:36.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:36.947+0000 7f603aa00640 1 -- 172.21.15.40:0/800643133 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6034058400 con 0x7f6044072060 2024-03-08T06:04:37.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.145+0000 7f6049000640 1 -- 172.21.15.40:0/800643133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f604406f280 con 0x7f6044072060 2024-03-08T06:04:37.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.146+0000 7f603aa00640 1 -- 172.21.15.40:0/800643133 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6034057d00 con 0x7f6044072060 2024-03-08T06:04:37.147 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:37.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 -- 172.21.15.40:0/800643133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f602006b4d0 msgr2=0x7f602006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:37.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f602006b4d0 0x7f602006d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f602c006fd0 tx=0x7f602c008040 comp rx=0 tx=0).stop 2024-03-08T06:04:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 -- 172.21.15.40:0/800643133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6044072060 msgr2=0x7f6044159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6044072060 0x7f6044159fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f603400c460 tx=0x7f603400c920 comp rx=0 tx=0).stop 2024-03-08T06:04:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 -- 172.21.15.40:0/800643133 shutdown_connections 2024-03-08T06:04:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f602006b4d0 0x7f602006d990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604415eb50 0x7f60440af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f60441584a0 0x7f604415b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 --2- 172.21.15.40:0/800643133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6044072060 0x7f6044159fe0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 -- 172.21.15.40:0/800643133 >> 172.21.15.40:0/800643133 conn(0x7f604406ae30 msgr2=0x7f6044162170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 -- 172.21.15.40:0/800643133 shutdown_connections 2024-03-08T06:04:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.148+0000 7f6049000640 1 -- 172.21.15.40:0/800643133 wait complete. 2024-03-08T06:04:37.169 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.390+0000 7f07ed800640 1 Processor -- start 2024-03-08T06:04:37.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.390+0000 7f07ed800640 1 -- start start 2024-03-08T06:04:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07ed800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f07e815de50 0x7f07e8161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07ed800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f07e80cf240 0x7f07e80d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07ed800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e8162520 con 0x7f07e80b97d0 2024-03-08T06:04:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07ed800640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e8162ae0 con 0x7f07e815de50 2024-03-08T06:04:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07ed800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e80d2360 con 0x7f07e80cf240 2024-03-08T06:04:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07e6a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f07e815de50 0x7f07e8161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07e6a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f07e815de50 0x7f07e8161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45224/0 (socket says 172.21.15.40:45224) 2024-03-08T06:04:37.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07e6a00640 1 -- 172.21.15.40:0/1038899633 learned_addr learned my addr 172.21.15.40:0/1038899633 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:37.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07e6a00640 1 -- 172.21.15.40:0/1038899633 >> v2:172.21.15.53:3300/0 conn(0x7f07e80cf240 msgr2=0x7f07e80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:37.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07e7400640 1 --2- 172.21.15.40:0/1038899633 >> v2:172.21.15.53:3300/0 conn(0x7f07e80cf240 0x7f07e80d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:37.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07e6a00640 1 --2- 172.21.15.40:0/1038899633 >> v2:172.21.15.53:3300/0 conn(0x7f07e80cf240 0x7f07e80d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07e6a00640 1 -- 172.21.15.40:0/1038899633 >> v1:172.21.15.53:6789/0 conn(0x7f07e80b97d0 legacy=0x7f07e80ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:37.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07e6a00640 1 -- 172.21.15.40:0/1038899633 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8162800 con 0x7f07e815de50 2024-03-08T06:04:37.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.392+0000 7f07e7400640 1 --2- 172.21.15.40:0/1038899633 >> v2:172.21.15.53:3300/0 conn(0x7f07e80cf240 0x7f07e80d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:37.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.393+0000 7f07e6a00640 1 --2- 172.21.15.40:0/1038899633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e815de50 0x7f07e8161bd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f07e80b68f0 tx=0x7f07d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6f327c5efc155f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.394+0000 7f07e5600640 1 -- 172.21.15.40:0/1038899633 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8035020 con 0x7f07e815de50 2024-03-08T06:04:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.394+0000 7f07e5600640 1 -- 172.21.15.40:0/1038899633 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d8031870 con 0x7f07e815de50 2024-03-08T06:04:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.394+0000 7f07e5600640 1 -- 172.21.15.40:0/1038899633 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8031c40 con 0x7f07e815de50 2024-03-08T06:04:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.394+0000 7f07ed800640 1 -- 172.21.15.40:0/1038899633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e815de50 msgr2=0x7f07e8161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.394+0000 7f07ed800640 1 --2- 172.21.15.40:0/1038899633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e815de50 0x7f07e8161bd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f07e80b68f0 tx=0x7f07d802b910 comp rx=0 tx=0).stop 2024-03-08T06:04:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.394+0000 7f07ed800640 1 -- 172.21.15.40:0/1038899633 shutdown_connections 2024-03-08T06:04:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.394+0000 7f07ed800640 1 --2- 172.21.15.40:0/1038899633 >> v2:172.21.15.53:3300/0 conn(0x7f07e80cf240 0x7f07e80d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.395+0000 7f07ed800640 1 --2- 172.21.15.40:0/1038899633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e815de50 0x7f07e8161bd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.395+0000 7f07ed800640 1 -- 172.21.15.40:0/1038899633 >> 172.21.15.40:0/1038899633 conn(0x7f07e806c490 msgr2=0x7f07e806c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.395+0000 7f07ed800640 1 -- 172.21.15.40:0/1038899633 shutdown_connections 2024-03-08T06:04:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.395+0000 7f07ed800640 1 -- 172.21.15.40:0/1038899633 wait complete. 2024-03-08T06:04:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.396+0000 7f07ed800640 1 Processor -- start 2024-03-08T06:04:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.396+0000 7f07ed800640 1 -- start start 2024-03-08T06:04:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.397+0000 7f07ed800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e80b97d0 0x7f07e80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.397+0000 7f07ed800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f07e80cf240 0x7f07e814d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.397+0000 7f07ed800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07e815de50 0x7f07e80c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.397+0000 7f07ed800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e8162290 con 0x7f07e80b97d0 2024-03-08T06:04:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.397+0000 7f07ed800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e8162110 con 0x7f07e815de50 2024-03-08T06:04:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.397+0000 7f07ed800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e815e230 con 0x7f07e80cf240 2024-03-08T06:04:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.398+0000 7f07e6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e80b97d0 0x7f07e80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.398+0000 7f07e6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e80b97d0 0x7f07e80cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45230/0 (socket says 172.21.15.40:45230) 2024-03-08T06:04:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.398+0000 7f07e6a00640 1 -- 172.21.15.40:0/211307215 learned_addr learned my addr 172.21.15.40:0/211307215 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.398+0000 7f07e6000640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f07e80cf240 0x7f07e814d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.398+0000 7f07e7400640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07e815de50 0x7f07e80c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:37.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.398+0000 7f07e6a00640 1 -- 172.21.15.40:0/211307215 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f07e80cf240 msgr2=0x7f07e814d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.398+0000 7f07e6a00640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f07e80cf240 0x7f07e814d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.398+0000 7f07e6a00640 1 -- 172.21.15.40:0/211307215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07e815de50 msgr2=0x7f07e80c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.398+0000 7f07e6a00640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07e815de50 0x7f07e80c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.398+0000 7f07e6a00640 1 -- 172.21.15.40:0/211307215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e814db30 con 0x7f07e80b97d0 2024-03-08T06:04:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.399+0000 7f07e6a00640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e80b97d0 0x7f07e80cd960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f07d8000c00 tx=0x7f07d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.399+0000 7f07dea00640 1 -- 172.21.15.40:0/211307215 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8040070 con 0x7f07e80b97d0 2024-03-08T06:04:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.399+0000 7f07dea00640 1 -- 172.21.15.40:0/211307215 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d803a420 con 0x7f07e80b97d0 2024-03-08T06:04:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.399+0000 7f07dea00640 1 -- 172.21.15.40:0/211307215 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8035040 con 0x7f07e80b97d0 2024-03-08T06:04:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.399+0000 7f07ed800640 1 -- 172.21.15.40:0/211307215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e80cc8c0 con 0x7f07e80b97d0 2024-03-08T06:04:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.399+0000 7f07ed800640 1 -- 172.21.15.40:0/211307215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e80cce80 con 0x7f07e80b97d0 2024-03-08T06:04:37.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.400+0000 7f07ed800640 1 -- 172.21.15.40:0/211307215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e8067cb0 con 0x7f07e80b97d0 2024-03-08T06:04:37.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.401+0000 7f07dea00640 1 -- 172.21.15.40:0/211307215 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07d804b060 con 0x7f07e80b97d0 2024-03-08T06:04:37.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.402+0000 7f07dea00640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f07c806b4d0 0x7f07c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:37.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.402+0000 7f07dea00640 1 -- 172.21.15.40:0/211307215 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07d80ac080 con 0x7f07e80b97d0 2024-03-08T06:04:37.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.402+0000 7f07e6000640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f07c806b4d0 0x7f07c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:37.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.403+0000 7f07e6000640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f07c806b4d0 0x7f07c806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f07d4007fd0 tx=0x7f07d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:37.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.407+0000 7f07dea00640 1 -- 172.21.15.40:0/211307215 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07d807af90 con 0x7f07e80b97d0 2024-03-08T06:04:37.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.605+0000 7f07ed800640 1 -- 172.21.15.40:0/211307215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f07e80d1a70 con 0x7f07e80b97d0 2024-03-08T06:04:37.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.606+0000 7f07dea00640 1 -- 172.21.15.40:0/211307215 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f07d807a890 con 0x7f07e80b97d0 2024-03-08T06:04:37.607 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.608+0000 7f07ed800640 1 -- 172.21.15.40:0/211307215 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f07c806b4d0 msgr2=0x7f07c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.608+0000 7f07ed800640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f07c806b4d0 0x7f07c806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f07d4007fd0 tx=0x7f07d4009040 comp rx=0 tx=0).stop 2024-03-08T06:04:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.608+0000 7f07ed800640 1 -- 172.21.15.40:0/211307215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e80b97d0 msgr2=0x7f07e80cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.608+0000 7f07ed800640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e80b97d0 0x7f07e80cd960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f07d8000c00 tx=0x7f07d80046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.609+0000 7f07ed800640 1 -- 172.21.15.40:0/211307215 shutdown_connections 2024-03-08T06:04:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.609+0000 7f07ed800640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f07c806b4d0 0x7f07c806d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.609+0000 7f07ed800640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07e815de50 0x7f07e80c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.609+0000 7f07ed800640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f07e80cf240 0x7f07e814d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.609+0000 7f07ed800640 1 --2- 172.21.15.40:0/211307215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e80b97d0 0x7f07e80cd960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.609+0000 7f07ed800640 1 -- 172.21.15.40:0/211307215 >> 172.21.15.40:0/211307215 conn(0x7f07e806c490 msgr2=0x7f07e815d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.609+0000 7f07ed800640 1 -- 172.21.15.40:0/211307215 shutdown_connections 2024-03-08T06:04:37.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.609+0000 7f07ed800640 1 -- 172.21.15.40:0/211307215 wait complete. 2024-03-08T06:04:37.628 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:37.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.846+0000 7fec35600640 1 Processor -- start 2024-03-08T06:04:37.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.847+0000 7fec35600640 1 -- start start 2024-03-08T06:04:37.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.848+0000 7fec35600640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec3015dce0 0x7fec300d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.848+0000 7fec35600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec300d25f0 con 0x7fec300d3690 2024-03-08T06:04:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.848+0000 7fec35600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec300d28d0 con 0x7fec3006c9a0 2024-03-08T06:04:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.848+0000 7fec35600640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec300d2bb0 con 0x7fec3015dce0 2024-03-08T06:04:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.848+0000 7fec2fe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec3015dce0 0x7fec300d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.849+0000 7fec2fe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fec3015dce0 0x7fec300d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45246/0 (socket says 172.21.15.40:45246) 2024-03-08T06:04:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.849+0000 7fec2fe00640 1 -- 172.21.15.40:0/3799091121 learned_addr learned my addr 172.21.15.40:0/3799091121 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.849+0000 7fec2f400640 1 --1- 172.21.15.40:0/3799091121 >> v1:172.21.15.40:6789/0 conn(0x7fec300d3690 0x7fec3006c290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42628/0 (socket says 172.21.15.40:42628) 2024-03-08T06:04:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.849+0000 7fec2fe00640 1 -- 172.21.15.40:0/3799091121 >> v1:172.21.15.53:6789/0 conn(0x7fec3006c9a0 legacy=0x7fec300b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.849+0000 7fec2fe00640 1 -- 172.21.15.40:0/3799091121 >> v1:172.21.15.40:6789/0 conn(0x7fec300d3690 legacy=0x7fec3006c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:37.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.849+0000 7fec2fe00640 1 -- 172.21.15.40:0/3799091121 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec300d2e90 con 0x7fec3015dce0 2024-03-08T06:04:37.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.850+0000 7fec2fe00640 1 --2- 172.21.15.40:0/3799091121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec3015dce0 0x7fec300d1fa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec20009870 tx=0x7fec2002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7865edbef62139f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:37.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.851+0000 7fec2ea00640 1 -- 172.21.15.40:0/3799091121 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20034070 con 0x7fec3015dce0 2024-03-08T06:04:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.851+0000 7fec2ea00640 1 -- 172.21.15.40:0/3799091121 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec20030860 con 0x7fec3015dce0 2024-03-08T06:04:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.851+0000 7fec2ea00640 1 -- 172.21.15.40:0/3799091121 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20030c30 con 0x7fec3015dce0 2024-03-08T06:04:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.851+0000 7fec35600640 1 -- 172.21.15.40:0/3799091121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec3015dce0 msgr2=0x7fec300d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.851+0000 7fec35600640 1 --2- 172.21.15.40:0/3799091121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec3015dce0 0x7fec300d1fa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec20009870 tx=0x7fec2002b9a0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.851+0000 7fec35600640 1 -- 172.21.15.40:0/3799091121 shutdown_connections 2024-03-08T06:04:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.851+0000 7fec35600640 1 --2- 172.21.15.40:0/3799091121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec3015dce0 0x7fec300d1fa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.851+0000 7fec35600640 1 -- 172.21.15.40:0/3799091121 >> 172.21.15.40:0/3799091121 conn(0x7fec30066fc0 msgr2=0x7fec300d3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.851+0000 7fec35600640 1 -- 172.21.15.40:0/3799091121 shutdown_connections 2024-03-08T06:04:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.851+0000 7fec35600640 1 -- 172.21.15.40:0/3799091121 wait complete. 2024-03-08T06:04:37.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.853+0000 7fec35600640 1 Processor -- start 2024-03-08T06:04:37.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.854+0000 7fec35600640 1 -- start start 2024-03-08T06:04:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.854+0000 7fec35600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fec3006c9a0 0x7fec30152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec35600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec300d3690 0x7fec301f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec35600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec3015dce0 0x7fec301f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec35600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3015e240 con 0x7fec300d3690 2024-03-08T06:04:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec35600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3015e0c0 con 0x7fec3015dce0 2024-03-08T06:04:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec35600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec300b9640 con 0x7fec3006c9a0 2024-03-08T06:04:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec2f400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec300d3690 0x7fec301f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec2f400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec300d3690 0x7fec301f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45248/0 (socket says 172.21.15.40:45248) 2024-03-08T06:04:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec2f400640 1 -- 172.21.15.40:0/520244473 learned_addr learned my addr 172.21.15.40:0/520244473 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec2fe00640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fec3006c9a0 0x7fec30152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec34c00640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec3015dce0 0x7fec301f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec2f400640 1 -- 172.21.15.40:0/520244473 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fec3006c9a0 msgr2=0x7fec30152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec2f400640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fec3006c9a0 0x7fec30152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec2f400640 1 -- 172.21.15.40:0/520244473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec3015dce0 msgr2=0x7fec301f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec2f400640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec3015dce0 0x7fec301f3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec2f400640 1 -- 172.21.15.40:0/520244473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec301f4d10 con 0x7fec300d3690 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec2fe00640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fec3006c9a0 0x7fec30152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.855+0000 7fec34c00640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec3015dce0 0x7fec301f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.856+0000 7fec2f400640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec300d3690 0x7fec301f47d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00c400 tx=0x7fec1c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.856+0000 7fec27e00640 1 -- 172.21.15.40:0/520244473 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c0074e0 con 0x7fec300d3690 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.856+0000 7fec27e00640 1 -- 172.21.15.40:0/520244473 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c007680 con 0x7fec300d3690 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.856+0000 7fec27e00640 1 -- 172.21.15.40:0/520244473 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c004a00 con 0x7fec300d3690 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.856+0000 7fec35600640 1 -- 172.21.15.40:0/520244473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec301f3bb0 con 0x7fec300d3690 2024-03-08T06:04:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.856+0000 7fec35600640 1 -- 172.21.15.40:0/520244473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec30159b50 con 0x7fec300d3690 2024-03-08T06:04:37.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.857+0000 7fec35600640 1 -- 172.21.15.40:0/520244473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec30075100 con 0x7fec300d3690 2024-03-08T06:04:37.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.858+0000 7fec27e00640 1 -- 172.21.15.40:0/520244473 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec1c01a020 con 0x7fec300d3690 2024-03-08T06:04:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.859+0000 7fec27e00640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fec1006b4d0 0x7fec1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.859+0000 7fec27e00640 1 -- 172.21.15.40:0/520244473 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec1c089920 con 0x7fec300d3690 2024-03-08T06:04:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.859+0000 7fec2fe00640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fec1006b4d0 0x7fec1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.860+0000 7fec2fe00640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fec1006b4d0 0x7fec1006d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec20036000 tx=0x7fec2003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:37.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:37.864+0000 7fec27e00640 1 -- 172.21.15.40:0/520244473 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec301f4d10 con 0x7fec300d3690 2024-03-08T06:04:38.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.064+0000 7fec35600640 1 -- 172.21.15.40:0/520244473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec3015ab80 con 0x7fec300d3690 2024-03-08T06:04:38.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.065+0000 7fec27e00640 1 -- 172.21.15.40:0/520244473 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7fec1c058830 con 0x7fec300d3690 2024-03-08T06:04:38.067 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:38.067 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:38.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 -- 172.21.15.40:0/520244473 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fec1006b4d0 msgr2=0x7fec1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fec1006b4d0 0x7fec1006d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec20036000 tx=0x7fec2003b040 comp rx=0 tx=0).stop 2024-03-08T06:04:38.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 -- 172.21.15.40:0/520244473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec300d3690 msgr2=0x7fec301f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec300d3690 0x7fec301f47d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00c400 tx=0x7fec1c00c8c0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 -- 172.21.15.40:0/520244473 shutdown_connections 2024-03-08T06:04:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fec1006b4d0 0x7fec1006d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec3015dce0 0x7fec301f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec300d3690 0x7fec301f47d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 --2- 172.21.15.40:0/520244473 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fec3006c9a0 0x7fec30152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 -- 172.21.15.40:0/520244473 >> 172.21.15.40:0/520244473 conn(0x7fec30066fc0 msgr2=0x7fec300b8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 -- 172.21.15.40:0/520244473 shutdown_connections 2024-03-08T06:04:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.068+0000 7fec35600640 1 -- 172.21.15.40:0/520244473 wait complete. 2024-03-08T06:04:38.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:38.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.310+0000 7fd5f9800640 1 Processor -- start 2024-03-08T06:04:38.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.310+0000 7fd5f9800640 1 -- start start 2024-03-08T06:04:38.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.312+0000 7fd5f9800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd5f415de60 0x7fd5f4161930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.312+0000 7fd5f9800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5f40b9630 0x7fd5f40ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.312+0000 7fd5f9800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f415e240 con 0x7fd5f40cee60 2024-03-08T06:04:38.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.312+0000 7fd5f9800640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f40cf2d0 con 0x7fd5f40b9630 2024-03-08T06:04:38.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.312+0000 7fd5f9800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f4162030 con 0x7fd5f415de60 2024-03-08T06:04:38.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.312+0000 7fd5f3400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5f40b9630 0x7fd5f40ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:38.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.312+0000 7fd5f3400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5f40b9630 0x7fd5f40ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45256/0 (socket says 172.21.15.40:45256) 2024-03-08T06:04:38.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.312+0000 7fd5f3400640 1 -- 172.21.15.40:0/1443634770 learned_addr learned my addr 172.21.15.40:0/1443634770 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:38.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.312+0000 7fd5f3e00640 1 --2- 172.21.15.40:0/1443634770 >> v2:172.21.15.53:3300/0 conn(0x7fd5f415de60 0x7fd5f4161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:38.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.313+0000 7fd5f3400640 1 -- 172.21.15.40:0/1443634770 >> v2:172.21.15.53:3300/0 conn(0x7fd5f415de60 msgr2=0x7fd5f4161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:38.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.313+0000 7fd5f3400640 1 --2- 172.21.15.40:0/1443634770 >> v2:172.21.15.53:3300/0 conn(0x7fd5f415de60 0x7fd5f4161930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.313+0000 7fd5f3400640 1 -- 172.21.15.40:0/1443634770 >> v1:172.21.15.53:6789/0 conn(0x7fd5f40cee60 legacy=0x7fd5f40b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:38.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.313+0000 7fd5f3400640 1 -- 172.21.15.40:0/1443634770 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f4162310 con 0x7fd5f40b9630 2024-03-08T06:04:38.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.313+0000 7fd5f3e00640 1 --2- 172.21.15.40:0/1443634770 >> v2:172.21.15.53:3300/0 conn(0x7fd5f415de60 0x7fd5f4161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:38.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.315+0000 7fd5f3400640 1 --2- 172.21.15.40:0/1443634770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5f40b9630 0x7fd5f40ce920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40b6740 tx=0x7fd5e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679faf66aaebd9aa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:38.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.315+0000 7fd5f2a00640 1 -- 172.21.15.40:0/1443634770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0035020 con 0x7fd5f40b9630 2024-03-08T06:04:38.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.315+0000 7fd5f2a00640 1 -- 172.21.15.40:0/1443634770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0031870 con 0x7fd5f40b9630 2024-03-08T06:04:38.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.315+0000 7fd5f2a00640 1 -- 172.21.15.40:0/1443634770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0031c40 con 0x7fd5f40b9630 2024-03-08T06:04:38.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.316+0000 7fd5f9800640 1 -- 172.21.15.40:0/1443634770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5f40b9630 msgr2=0x7fd5f40ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.316+0000 7fd5f9800640 1 --2- 172.21.15.40:0/1443634770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5f40b9630 0x7fd5f40ce920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40b6740 tx=0x7fd5e002b910 comp rx=0 tx=0).stop 2024-03-08T06:04:38.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.316+0000 7fd5f9800640 1 -- 172.21.15.40:0/1443634770 shutdown_connections 2024-03-08T06:04:38.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.316+0000 7fd5f9800640 1 --2- 172.21.15.40:0/1443634770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5f40b9630 0x7fd5f40ce920 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.316+0000 7fd5f9800640 1 --2- 172.21.15.40:0/1443634770 >> v2:172.21.15.53:3300/0 conn(0x7fd5f415de60 0x7fd5f4161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.316+0000 7fd5f9800640 1 -- 172.21.15.40:0/1443634770 >> 172.21.15.40:0/1443634770 conn(0x7fd5f406c500 msgr2=0x7fd5f406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:38.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.316+0000 7fd5f9800640 1 -- 172.21.15.40:0/1443634770 shutdown_connections 2024-03-08T06:04:38.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.316+0000 7fd5f9800640 1 -- 172.21.15.40:0/1443634770 wait complete. 2024-03-08T06:04:38.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.318+0000 7fd5f9800640 1 Processor -- start 2024-03-08T06:04:38.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.318+0000 7fd5f9800640 1 -- start start 2024-03-08T06:04:38.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f9800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5f40b9630 0x7fd5f4146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f9800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5f40cee60 0x7fd5f40c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f9800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f415de60 0x7fd5f4158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f9800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f40b9a90 con 0x7fd5f40cee60 2024-03-08T06:04:38.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f9800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f40d20c0 con 0x7fd5f415de60 2024-03-08T06:04:38.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f9800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f40d35f0 con 0x7fd5f40b9630 2024-03-08T06:04:38.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f3400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5f40cee60 0x7fd5f40c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:38.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f3400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5f40cee60 0x7fd5f40c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45266/0 (socket says 172.21.15.40:45266) 2024-03-08T06:04:38.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f3400640 1 -- 172.21.15.40:0/385511613 learned_addr learned my addr 172.21.15.40:0/385511613 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:38.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f3400640 1 -- 172.21.15.40:0/385511613 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5f40b9630 msgr2=0x7fd5f4146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f3400640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5f40b9630 0x7fd5f4146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f3400640 1 -- 172.21.15.40:0/385511613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f415de60 msgr2=0x7fd5f4158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f3400640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f415de60 0x7fd5f4158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.319+0000 7fd5f3400640 1 -- 172.21.15.40:0/385511613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f40ca4b0 con 0x7fd5f40cee60 2024-03-08T06:04:38.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.320+0000 7fd5f3400640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5f40cee60 0x7fd5f40c9f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd5e002b950 tx=0x7fd5e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:38.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.320+0000 7fd5ebe00640 1 -- 172.21.15.40:0/385511613 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0040070 con 0x7fd5f40cee60 2024-03-08T06:04:38.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.320+0000 7fd5ebe00640 1 -- 172.21.15.40:0/385511613 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0030d00 con 0x7fd5f40cee60 2024-03-08T06:04:38.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.320+0000 7fd5ebe00640 1 -- 172.21.15.40:0/385511613 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0035040 con 0x7fd5f40cee60 2024-03-08T06:04:38.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.320+0000 7fd5f9800640 1 -- 172.21.15.40:0/385511613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f41472f0 con 0x7fd5f40cee60 2024-03-08T06:04:38.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.320+0000 7fd5f9800640 1 -- 172.21.15.40:0/385511613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f41478b0 con 0x7fd5f40cee60 2024-03-08T06:04:38.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.322+0000 7fd5f9800640 1 -- 172.21.15.40:0/385511613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f4075090 con 0x7fd5f40cee60 2024-03-08T06:04:38.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.323+0000 7fd5ebe00640 1 -- 172.21.15.40:0/385511613 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5e0038020 con 0x7fd5f40cee60 2024-03-08T06:04:38.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.323+0000 7fd5ebe00640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d406b4d0 0x7fd5d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.324+0000 7fd5ebe00640 1 -- 172.21.15.40:0/385511613 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5e00ac110 con 0x7fd5f40cee60 2024-03-08T06:04:38.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.324+0000 7fd5f3e00640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d406b4d0 0x7fd5d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:38.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.324+0000 7fd5f3e00640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d406b4d0 0x7fd5d406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006fd0 tx=0x7fd5e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:38.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.328+0000 7fd5ebe00640 1 -- 172.21.15.40:0/385511613 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5e007aef0 con 0x7fd5f40cee60 2024-03-08T06:04:38.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.517+0000 7fd5f9800640 1 -- 172.21.15.40:0/385511613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5f40d17f0 con 0x7fd5f40cee60 2024-03-08T06:04:38.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.518+0000 7fd5ebe00640 1 -- 172.21.15.40:0/385511613 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5e007a7f0 con 0x7fd5f40cee60 2024-03-08T06:04:38.519 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.520+0000 7fd5f9800640 1 -- 172.21.15.40:0/385511613 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d406b4d0 msgr2=0x7fd5d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.520+0000 7fd5f9800640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d406b4d0 0x7fd5d406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006fd0 tx=0x7fd5e4008040 comp rx=0 tx=0).stop 2024-03-08T06:04:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.520+0000 7fd5f9800640 1 -- 172.21.15.40:0/385511613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5f40cee60 msgr2=0x7fd5f40c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.520+0000 7fd5f9800640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5f40cee60 0x7fd5f40c9f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd5e002b950 tx=0x7fd5e00046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.521+0000 7fd5f9800640 1 -- 172.21.15.40:0/385511613 shutdown_connections 2024-03-08T06:04:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.521+0000 7fd5f9800640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd5d406b4d0 0x7fd5d406d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.521+0000 7fd5f9800640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f415de60 0x7fd5f4158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.521+0000 7fd5f9800640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5f40cee60 0x7fd5f40c9f70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.521+0000 7fd5f9800640 1 --2- 172.21.15.40:0/385511613 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd5f40b9630 0x7fd5f4146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.521+0000 7fd5f9800640 1 -- 172.21.15.40:0/385511613 >> 172.21.15.40:0/385511613 conn(0x7fd5f406c500 msgr2=0x7fd5f406c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.521+0000 7fd5f9800640 1 -- 172.21.15.40:0/385511613 shutdown_connections 2024-03-08T06:04:38.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.521+0000 7fd5f9800640 1 -- 172.21.15.40:0/385511613 wait complete. 2024-03-08T06:04:38.541 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:38.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.764+0000 7f046f200640 1 Processor -- start 2024-03-08T06:04:38.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.764+0000 7f046f200640 1 -- start start 2024-03-08T06:04:38.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.766+0000 7f046f200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0468071fa0 0x7f04680cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.766+0000 7f046f200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f046815e760 0x7f0468144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.766+0000 7f046f200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046815ebf0 con 0x7f0468071bc0 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.766+0000 7f046f200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046815ee70 con 0x7f0468071fa0 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.766+0000 7f046f200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046815f150 con 0x7f046815e760 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.766+0000 7f0467400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0468071fa0 0x7f04680cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.766+0000 7f0467400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0468071fa0 0x7f04680cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45270/0 (socket says 172.21.15.40:45270) 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.766+0000 7f0467400640 1 -- 172.21.15.40:0/4013423535 learned_addr learned my addr 172.21.15.40:0/4013423535 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.766+0000 7f046d200640 1 --2- 172.21.15.40:0/4013423535 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f046815e760 0x7f0468144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.767+0000 7f0467400640 1 -- 172.21.15.40:0/4013423535 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f046815e760 msgr2=0x7f0468144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.767+0000 7f0467400640 1 --2- 172.21.15.40:0/4013423535 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f046815e760 0x7f0468144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.767+0000 7f0467400640 1 -- 172.21.15.40:0/4013423535 >> v1:172.21.15.53:6789/0 conn(0x7f0468071bc0 legacy=0x7f046815b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:38.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.767+0000 7f0467400640 1 -- 172.21.15.40:0/4013423535 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046815f430 con 0x7f0468071fa0 2024-03-08T06:04:38.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.768+0000 7f0467400640 1 --2- 172.21.15.40:0/4013423535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0468071fa0 0x7f04680cf5b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0450009870 tx=0x7f045002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adecf32f06f2182e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:38.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.768+0000 7f0466a00640 1 -- 172.21.15.40:0/4013423535 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450035020 con 0x7f0468071fa0 2024-03-08T06:04:38.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.768+0000 7f0466a00640 1 -- 172.21.15.40:0/4013423535 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0450031640 con 0x7f0468071fa0 2024-03-08T06:04:38.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.768+0000 7f0466a00640 1 -- 172.21.15.40:0/4013423535 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045002d040 con 0x7f0468071fa0 2024-03-08T06:04:38.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.769+0000 7f046f200640 1 -- 172.21.15.40:0/4013423535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0468071fa0 msgr2=0x7f04680cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.769+0000 7f046f200640 1 --2- 172.21.15.40:0/4013423535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0468071fa0 0x7f04680cf5b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0450009870 tx=0x7f045002b8f0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.769+0000 7f046f200640 1 -- 172.21.15.40:0/4013423535 shutdown_connections 2024-03-08T06:04:38.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.769+0000 7f046f200640 1 --2- 172.21.15.40:0/4013423535 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f046815e760 0x7f0468144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.769+0000 7f046f200640 1 --2- 172.21.15.40:0/4013423535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0468071fa0 0x7f04680cf5b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.769+0000 7f046f200640 1 -- 172.21.15.40:0/4013423535 >> 172.21.15.40:0/4013423535 conn(0x7f046806ae30 msgr2=0x7f046806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:38.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.769+0000 7f046f200640 1 -- 172.21.15.40:0/4013423535 shutdown_connections 2024-03-08T06:04:38.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.769+0000 7f046f200640 1 -- 172.21.15.40:0/4013423535 wait complete. 2024-03-08T06:04:38.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.771+0000 7f046f200640 1 Processor -- start 2024-03-08T06:04:38.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.771+0000 7f046f200640 1 -- start start 2024-03-08T06:04:38.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0468071bc0 0x7f0468156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046f200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0468071fa0 0x7f04680cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046f200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815e760 0x7f04680cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046f200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04681448d0 con 0x7f046815e760 2024-03-08T06:04:38.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0468144750 con 0x7f0468071bc0 2024-03-08T06:04:38.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046f200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04680b3fd0 con 0x7f0468071fa0 2024-03-08T06:04:38.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046d200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815e760 0x7f04680cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:38.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046d200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815e760 0x7f04680cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45274/0 (socket says 172.21.15.40:45274) 2024-03-08T06:04:38.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046d200640 1 -- 172.21.15.40:0/529103317 learned_addr learned my addr 172.21.15.40:0/529103317 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046d200640 1 -- 172.21.15.40:0/529103317 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0468071fa0 msgr2=0x7f04680cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046d200640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0468071fa0 0x7f04680cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046d200640 1 -- 172.21.15.40:0/529103317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0468071bc0 msgr2=0x7f0468156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f0467e00640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0468071bc0 0x7f0468156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046d200640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0468071bc0 0x7f0468156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.772+0000 7f046d200640 1 -- 172.21.15.40:0/529103317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046815a780 con 0x7f046815e760 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.773+0000 7f0467e00640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0468071bc0 0x7f0468156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.773+0000 7f046d200640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815e760 0x7f04680cc700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f046000d600 tx=0x7f046000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.773+0000 7f045fe00640 1 -- 172.21.15.40:0/529103317 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600125f0 con 0x7f046815e760 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.773+0000 7f045fe00640 1 -- 172.21.15.40:0/529103317 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0460019070 con 0x7f046815e760 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.773+0000 7f046f200640 1 -- 172.21.15.40:0/529103317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046815aa60 con 0x7f046815e760 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.773+0000 7f045fe00640 1 -- 172.21.15.40:0/529103317 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600129f0 con 0x7f046815e760 2024-03-08T06:04:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.773+0000 7f046f200640 1 -- 172.21.15.40:0/529103317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046815b020 con 0x7f046815e760 2024-03-08T06:04:38.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.774+0000 7f046f200640 1 -- 172.21.15.40:0/529103317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0468067cb0 con 0x7f046815e760 2024-03-08T06:04:38.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.775+0000 7f045fe00640 1 -- 172.21.15.40:0/529103317 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0460012b90 con 0x7f046815e760 2024-03-08T06:04:38.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.776+0000 7f045fe00640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f044406b500 0x7f044406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:38.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.776+0000 7f045fe00640 1 -- 172.21.15.40:0/529103317 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f046008a670 con 0x7f046815e760 2024-03-08T06:04:38.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.776+0000 7f0467e00640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f044406b500 0x7f044406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:38.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.777+0000 7f0467e00640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f044406b500 0x7f044406d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f04580012e0 tx=0x7f0458009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:38.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.781+0000 7f045fe00640 1 -- 172.21.15.40:0/529103317 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04600593d0 con 0x7f046815e760 2024-03-08T06:04:38.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.976+0000 7f046f200640 1 -- 172.21.15.40:0/529103317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04680b2d90 con 0x7f046815e760 2024-03-08T06:04:38.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.977+0000 7f045fe00640 1 -- 172.21.15.40:0/529103317 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0460058cd0 con 0x7f046815e760 2024-03-08T06:04:38.978 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.979+0000 7f046f200640 1 -- 172.21.15.40:0/529103317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f044406b500 msgr2=0x7f044406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.979+0000 7f046f200640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f044406b500 0x7f044406d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f04580012e0 tx=0x7f0458009040 comp rx=0 tx=0).stop 2024-03-08T06:04:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.979+0000 7f046f200640 1 -- 172.21.15.40:0/529103317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815e760 msgr2=0x7f04680cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.979+0000 7f046f200640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815e760 0x7f04680cc700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f046000d600 tx=0x7f046000dac0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.980+0000 7f046f200640 1 -- 172.21.15.40:0/529103317 shutdown_connections 2024-03-08T06:04:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.980+0000 7f046f200640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f044406b500 0x7f044406d9c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.980+0000 7f046f200640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f046815e760 0x7f04680cc700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.980+0000 7f046f200640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0468071fa0 0x7f04680cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.980+0000 7f046f200640 1 --2- 172.21.15.40:0/529103317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0468071bc0 0x7f0468156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.980+0000 7f046f200640 1 -- 172.21.15.40:0/529103317 >> 172.21.15.40:0/529103317 conn(0x7f046806ae30 msgr2=0x7f04680b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.980+0000 7f046f200640 1 -- 172.21.15.40:0/529103317 shutdown_connections 2024-03-08T06:04:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:38.980+0000 7f046f200640 1 -- 172.21.15.40:0/529103317 wait complete. 2024-03-08T06:04:39.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:39.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.223+0000 7f342ac00640 1 Processor -- start 2024-03-08T06:04:39.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.224+0000 7f342ac00640 1 -- start start 2024-03-08T06:04:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.225+0000 7f342ac00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3424145500 0x7f342406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.225+0000 7f342ac00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f342415bfb0 0x7f34240716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.225+0000 7f342ac00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3424071c20 0x7f342406d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.225+0000 7f342ac00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3424072050 con 0x7f3424071c20 2024-03-08T06:04:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.225+0000 7f342ac00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342415c460 con 0x7f342415bfb0 2024-03-08T06:04:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.225+0000 7f342ac00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342415c740 con 0x7f3424145500 2024-03-08T06:04:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.226+0000 7f342a200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3424071c20 0x7f342406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.226+0000 7f342a200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3424071c20 0x7f342406d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45290/0 (socket says 172.21.15.40:45290) 2024-03-08T06:04:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.226+0000 7f342a200640 1 -- 172.21.15.40:0/3299053663 learned_addr learned my addr 172.21.15.40:0/3299053663 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.226+0000 7f3429800640 1 --2- 172.21.15.40:0/3299053663 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3424145500 0x7f342406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.226+0000 7f3428e00640 1 --2- 172.21.15.40:0/3299053663 >> v2:172.21.15.53:3300/0 conn(0x7f342415bfb0 0x7f34240716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:39.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.226+0000 7f342a200640 1 -- 172.21.15.40:0/3299053663 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3424145500 msgr2=0x7f342406ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:39.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.226+0000 7f342a200640 1 --2- 172.21.15.40:0/3299053663 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3424145500 0x7f342406ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.226+0000 7f342a200640 1 -- 172.21.15.40:0/3299053663 >> v2:172.21.15.53:3300/0 conn(0x7f342415bfb0 msgr2=0x7f34240716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:39.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.226+0000 7f342a200640 1 --2- 172.21.15.40:0/3299053663 >> v2:172.21.15.53:3300/0 conn(0x7f342415bfb0 0x7f34240716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.226+0000 7f342a200640 1 -- 172.21.15.40:0/3299053663 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342415ca20 con 0x7f3424071c20 2024-03-08T06:04:39.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.227+0000 7f342a200640 1 --2- 172.21.15.40:0/3299053663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3424071c20 0x7f342406d220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f342406f9b0 tx=0x7f341c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb5e7a6e0222cb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.228+0000 7f3423e00640 1 -- 172.21.15.40:0/3299053663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c02c9e0 con 0x7f3424071c20 2024-03-08T06:04:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.228+0000 7f3423e00640 1 -- 172.21.15.40:0/3299053663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c02cb80 con 0x7f3424071c20 2024-03-08T06:04:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.228+0000 7f3423e00640 1 -- 172.21.15.40:0/3299053663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c0343f0 con 0x7f3424071c20 2024-03-08T06:04:39.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.228+0000 7f342ac00640 1 -- 172.21.15.40:0/3299053663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3424071c20 msgr2=0x7f342406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:39.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.228+0000 7f342ac00640 1 --2- 172.21.15.40:0/3299053663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3424071c20 0x7f342406d220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f342406f9b0 tx=0x7f341c02be40 comp rx=0 tx=0).stop 2024-03-08T06:04:39.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.229+0000 7f342ac00640 1 -- 172.21.15.40:0/3299053663 shutdown_connections 2024-03-08T06:04:39.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.229+0000 7f342ac00640 1 --2- 172.21.15.40:0/3299053663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3424071c20 0x7f342406d220 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.229+0000 7f342ac00640 1 --2- 172.21.15.40:0/3299053663 >> v2:172.21.15.53:3300/0 conn(0x7f342415bfb0 0x7f34240716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.229+0000 7f342ac00640 1 --2- 172.21.15.40:0/3299053663 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3424145500 0x7f342406ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.229+0000 7f342ac00640 1 -- 172.21.15.40:0/3299053663 >> 172.21.15.40:0/3299053663 conn(0x7f34240b1850 msgr2=0x7f34240b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:39.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.229+0000 7f342ac00640 1 -- 172.21.15.40:0/3299053663 shutdown_connections 2024-03-08T06:04:39.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.229+0000 7f342ac00640 1 -- 172.21.15.40:0/3299053663 wait complete. 2024-03-08T06:04:39.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.230+0000 7f342ac00640 1 Processor -- start 2024-03-08T06:04:39.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.231+0000 7f342ac00640 1 -- start start 2024-03-08T06:04:39.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342ac00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3424071c20 0x7f3424140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342ac00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3424145500 0x7f3424157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342ac00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342415bfb0 0x7f34240adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342ac00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342406f370 con 0x7f342415bfb0 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342ac00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342406f1f0 con 0x7f3424145500 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342ac00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342406d760 con 0x7f3424071c20 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342a200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342415bfb0 0x7f34240adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342a200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342415bfb0 0x7f34240adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45292/0 (socket says 172.21.15.40:45292) 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342a200640 1 -- 172.21.15.40:0/2034522315 learned_addr learned my addr 172.21.15.40:0/2034522315 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f3428e00640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3424145500 0x7f3424157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f3429800640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3424071c20 0x7f3424140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342a200640 1 -- 172.21.15.40:0/2034522315 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3424071c20 msgr2=0x7f3424140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342a200640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3424071c20 0x7f3424140f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342a200640 1 -- 172.21.15.40:0/2034522315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3424145500 msgr2=0x7f3424157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342a200640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3424145500 0x7f3424157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f342a200640 1 -- 172.21.15.40:0/2034522315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424158460 con 0x7f342415bfb0 2024-03-08T06:04:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.232+0000 7f3428e00640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3424145500 0x7f3424157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:39.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.233+0000 7f342a200640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342415bfb0 0x7f34240adef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f341c036040 tx=0x7f341c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:39.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.233+0000 7f3421600640 1 -- 172.21.15.40:0/2034522315 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c043020 con 0x7f342415bfb0 2024-03-08T06:04:39.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.233+0000 7f3421600640 1 -- 172.21.15.40:0/2034522315 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c03e450 con 0x7f342415bfb0 2024-03-08T06:04:39.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.233+0000 7f3421600640 1 -- 172.21.15.40:0/2034522315 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c009510 con 0x7f342415bfb0 2024-03-08T06:04:39.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.233+0000 7f342ac00640 1 -- 172.21.15.40:0/2034522315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3424158740 con 0x7f342415bfb0 2024-03-08T06:04:39.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.233+0000 7f342ac00640 1 -- 172.21.15.40:0/2034522315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3424140a00 con 0x7f342415bfb0 2024-03-08T06:04:39.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.234+0000 7f342ac00640 1 -- 172.21.15.40:0/2034522315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3424075090 con 0x7f342415bfb0 2024-03-08T06:04:39.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.235+0000 7f3421600640 1 -- 172.21.15.40:0/2034522315 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f341c046020 con 0x7f342415bfb0 2024-03-08T06:04:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.236+0000 7f3421600640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f340406b500 0x7f340406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.236+0000 7f3421600640 1 -- 172.21.15.40:0/2034522315 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f341c0ac700 con 0x7f342415bfb0 2024-03-08T06:04:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.236+0000 7f3429800640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f340406b500 0x7f340406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.237+0000 7f3429800640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f340406b500 0x7f340406d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3418006fd0 tx=0x7f3418008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:39.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.241+0000 7f3421600640 1 -- 172.21.15.40:0/2034522315 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f341c04d070 con 0x7f342415bfb0 2024-03-08T06:04:39.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.438+0000 7f342ac00640 1 -- 172.21.15.40:0/2034522315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34240ce460 con 0x7f342415bfb0 2024-03-08T06:04:39.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.439+0000 7f3421600640 1 -- 172.21.15.40:0/2034522315 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f341c07b570 con 0x7f342415bfb0 2024-03-08T06:04:39.440 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 -- 172.21.15.40:0/2034522315 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f340406b500 msgr2=0x7f340406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f340406b500 0x7f340406d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3418006fd0 tx=0x7f3418008040 comp rx=0 tx=0).stop 2024-03-08T06:04:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 -- 172.21.15.40:0/2034522315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342415bfb0 msgr2=0x7f34240adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342415bfb0 0x7f34240adef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f341c036040 tx=0x7f341c004a40 comp rx=0 tx=0).stop 2024-03-08T06:04:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 -- 172.21.15.40:0/2034522315 shutdown_connections 2024-03-08T06:04:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f340406b500 0x7f340406d9c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342415bfb0 0x7f34240adef0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3424145500 0x7f3424157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 --2- 172.21.15.40:0/2034522315 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3424071c20 0x7f3424140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 -- 172.21.15.40:0/2034522315 >> 172.21.15.40:0/2034522315 conn(0x7f34240b1850 msgr2=0x7f34240afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 -- 172.21.15.40:0/2034522315 shutdown_connections 2024-03-08T06:04:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.442+0000 7f342ac00640 1 -- 172.21.15.40:0/2034522315 wait complete. 2024-03-08T06:04:39.482 INFO:tasks.cephfs_test_runner:Starting test: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) That the per-client labelled perf counters depict the clients 2024-03-08T06:04:39.482 INFO:tasks.cephfs_test_runner:test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T06:04:39.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:39.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.706+0000 7fc7c5200640 1 Processor -- start 2024-03-08T06:04:39.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.706+0000 7fc7c5200640 1 -- start start 2024-03-08T06:04:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.707+0000 7fc7c5200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7c0099c30 0x7fc7c00b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.707+0000 7fc7c5200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7c006c770 0x7fc7c009a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.707+0000 7fc7c5200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c009a550 con 0x7fc7c00cbfe0 2024-03-08T06:04:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7c5200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c015f450 con 0x7fc7c006c770 2024-03-08T06:04:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7c5200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c006cee0 con 0x7fc7c0099c30 2024-03-08T06:04:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7bea00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7c006c770 0x7fc7c009a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7bea00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7c006c770 0x7fc7c009a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45300/0 (socket says 172.21.15.40:45300) 2024-03-08T06:04:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7bfe00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc7c00cbfe0 0x7fc7c00d09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42632/0 (socket says 172.21.15.40:42632) 2024-03-08T06:04:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7bea00640 1 -- 172.21.15.40:0/4042356259 learned_addr learned my addr 172.21.15.40:0/4042356259 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:39.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7bf400640 1 --2- 172.21.15.40:0/4042356259 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7c0099c30 0x7fc7c00b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:39.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7bea00640 1 -- 172.21.15.40:0/4042356259 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7c0099c30 msgr2=0x7fc7c00b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:39.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7bea00640 1 --2- 172.21.15.40:0/4042356259 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7c0099c30 0x7fc7c00b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7bea00640 1 -- 172.21.15.40:0/4042356259 >> v1:172.21.15.40:6789/0 conn(0x7fc7c00cbfe0 legacy=0x7fc7c00d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:39.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7bea00640 1 -- 172.21.15.40:0/4042356259 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c00cff00 con 0x7fc7c006c770 2024-03-08T06:04:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.708+0000 7fc7be000640 1 -- 172.21.15.40:0/4042356259 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620970180 0 0) 0x7fc7c009a550 con 0x7fc7c00cbfe0 2024-03-08T06:04:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.710+0000 7fc7bea00640 1 --2- 172.21.15.40:0/4042356259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c006c770 0x7fc7c009a010 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009e50 tx=0x7fc7b002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a22ed436dab844 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.710+0000 7fc7be000640 1 -- 172.21.15.40:0/4042356259 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b002cb00 con 0x7fc7c006c770 2024-03-08T06:04:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.710+0000 7fc7be000640 1 -- 172.21.15.40:0/4042356259 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b002cca0 con 0x7fc7c006c770 2024-03-08T06:04:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.710+0000 7fc7be000640 1 -- 172.21.15.40:0/4042356259 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0034610 con 0x7fc7c006c770 2024-03-08T06:04:39.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.711+0000 7fc7c5200640 1 -- 172.21.15.40:0/4042356259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c006c770 msgr2=0x7fc7c009a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:39.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.711+0000 7fc7c5200640 1 --2- 172.21.15.40:0/4042356259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c006c770 0x7fc7c009a010 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009e50 tx=0x7fc7b002bf60 comp rx=0 tx=0).stop 2024-03-08T06:04:39.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.711+0000 7fc7c5200640 1 -- 172.21.15.40:0/4042356259 shutdown_connections 2024-03-08T06:04:39.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.711+0000 7fc7c5200640 1 --2- 172.21.15.40:0/4042356259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c006c770 0x7fc7c009a010 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.711+0000 7fc7c5200640 1 --2- 172.21.15.40:0/4042356259 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7c0099c30 0x7fc7c00b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.711+0000 7fc7c5200640 1 -- 172.21.15.40:0/4042356259 >> 172.21.15.40:0/4042356259 conn(0x7fc7c00b3480 msgr2=0x7fc7c00b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:39.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.711+0000 7fc7c5200640 1 -- 172.21.15.40:0/4042356259 shutdown_connections 2024-03-08T06:04:39.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.711+0000 7fc7c5200640 1 -- 172.21.15.40:0/4042356259 wait complete. 2024-03-08T06:04:39.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.713+0000 7fc7c5200640 1 Processor -- start 2024-03-08T06:04:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.713+0000 7fc7c5200640 1 -- start start 2024-03-08T06:04:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.714+0000 7fc7c5200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c006c770 0x7fc7c01f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.714+0000 7fc7c5200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7c0099c30 0x7fc7c015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.714+0000 7fc7c5200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7c00cbfe0 0x7fc7c01f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.714+0000 7fc7c5200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c006cbf0 con 0x7fc7c006c770 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.714+0000 7fc7c5200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00d04c0 con 0x7fc7c00cbfe0 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.714+0000 7fc7c5200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00cc470 con 0x7fc7c0099c30 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.714+0000 7fc7bf400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c006c770 0x7fc7c01f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.714+0000 7fc7bf400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c006c770 0x7fc7c01f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45308/0 (socket says 172.21.15.40:45308) 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.714+0000 7fc7bf400640 1 -- 172.21.15.40:0/892384344 learned_addr learned my addr 172.21.15.40:0/892384344 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.714+0000 7fc7bf400640 1 -- 172.21.15.40:0/892384344 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7c0099c30 msgr2=0x7fc7c015aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.715+0000 7fc7bf400640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7c0099c30 0x7fc7c015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.715+0000 7fc7bf400640 1 -- 172.21.15.40:0/892384344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7c00cbfe0 msgr2=0x7fc7c01f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.715+0000 7fc7bf400640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7c00cbfe0 0x7fc7c01f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.715+0000 7fc7bf400640 1 -- 172.21.15.40:0/892384344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c01f7de0 con 0x7fc7c006c770 2024-03-08T06:04:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.715+0000 7fc7bf400640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c006c770 0x7fc7c01f47d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800d5f0 tx=0x7fc7a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.715+0000 7fc7b7400640 1 -- 172.21.15.40:0/892384344 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80125f0 con 0x7fc7c006c770 2024-03-08T06:04:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.715+0000 7fc7b7400640 1 -- 172.21.15.40:0/892384344 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a8019070 con 0x7fc7c006c770 2024-03-08T06:04:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.715+0000 7fc7b7400640 1 -- 172.21.15.40:0/892384344 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80129d0 con 0x7fc7c006c770 2024-03-08T06:04:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.715+0000 7fc7c5200640 1 -- 172.21.15.40:0/892384344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c01f80c0 con 0x7fc7c006c770 2024-03-08T06:04:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.715+0000 7fc7c5200640 1 -- 172.21.15.40:0/892384344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c00b6fa0 con 0x7fc7c006c770 2024-03-08T06:04:39.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.717+0000 7fc7c5200640 1 -- 172.21.15.40:0/892384344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c0075100 con 0x7fc7c006c770 2024-03-08T06:04:39.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.717+0000 7fc7b7400640 1 -- 172.21.15.40:0/892384344 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7a8012b70 con 0x7fc7c006c770 2024-03-08T06:04:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.718+0000 7fc7b7400640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc79c06b520 0x7fc79c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.718+0000 7fc7b7400640 1 -- 172.21.15.40:0/892384344 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7a808a640 con 0x7fc7c006c770 2024-03-08T06:04:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.719+0000 7fc7bea00640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc79c06b520 0x7fc79c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:39.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.719+0000 7fc7bea00640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc79c06b520 0x7fc79c06d9e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0035000 tx=0x7fc7b002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:39.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.723+0000 7fc7b7400640 1 -- 172.21.15.40:0/892384344 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7a80594a0 con 0x7fc7c006c770 2024-03-08T06:04:39.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.924+0000 7fc7c5200640 1 -- 172.21.15.40:0/892384344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7c00b7450 con 0x7fc7c006c770 2024-03-08T06:04:39.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.927+0000 7fc7b7400640 1 -- 172.21.15.40:0/892384344 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7fc7a8058da0 con 0x7fc7c006c770 2024-03-08T06:04:39.928 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:39.928 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:39.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 -- 172.21.15.40:0/892384344 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc79c06b520 msgr2=0x7fc79c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:39.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc79c06b520 0x7fc79c06d9e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0035000 tx=0x7fc7b002e000 comp rx=0 tx=0).stop 2024-03-08T06:04:39.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 -- 172.21.15.40:0/892384344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c006c770 msgr2=0x7fc7c01f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:39.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c006c770 0x7fc7c01f47d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800d5f0 tx=0x7fc7a800dab0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 -- 172.21.15.40:0/892384344 shutdown_connections 2024-03-08T06:04:39.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc79c06b520 0x7fc79c06d9e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7c00cbfe0 0x7fc7c01f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7c0099c30 0x7fc7c015aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 --2- 172.21.15.40:0/892384344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c006c770 0x7fc7c01f47d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:39.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 -- 172.21.15.40:0/892384344 >> 172.21.15.40:0/892384344 conn(0x7fc7c00b3480 msgr2=0x7fc7c015e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:39.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 -- 172.21.15.40:0/892384344 shutdown_connections 2024-03-08T06:04:39.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:39.930+0000 7fc7c5200640 1 -- 172.21.15.40:0/892384344 wait complete. 2024-03-08T06:04:39.951 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:39.951 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.173+0000 7f7b11000640 1 Processor -- start 2024-03-08T06:04:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.173+0000 7f7b11000640 1 -- start start 2024-03-08T06:04:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.175+0000 7f7b11000640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7b0c145500 0x7f7b0c06ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.175+0000 7f7b11000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c15c460 con 0x7f7b0c071df0 2024-03-08T06:04:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.175+0000 7f7b11000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c15c740 con 0x7f7b0c15bfb0 2024-03-08T06:04:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.175+0000 7f7b11000640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0c15ca20 con 0x7f7b0c145500 2024-03-08T06:04:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.175+0000 7f7b0be00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7b0c071df0 0x7f7b0c0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42638/0 (socket says 172.21.15.40:42638) 2024-03-08T06:04:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.175+0000 7f7b0be00640 1 -- 172.21.15.40:0/737650064 learned_addr learned my addr 172.21.15.40:0/737650064 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:40.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238504905 0 0) 0x7f7b0c15c460 con 0x7f7b0c071df0 2024-03-08T06:04:40.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0b400640 1 --2- 172.21.15.40:0/737650064 >> v2:172.21.15.53:3300/0 conn(0x7f7b0c145500 0x7f7b0c06ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7af0003520 con 0x7f7b0c071df0 2024-03-08T06:04:40.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 708661154 0 0) 0x7f7af0003520 con 0x7f7b0c071df0 2024-03-08T06:04:40.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 >> v2:172.21.15.53:3300/0 conn(0x7f7b0c145500 msgr2=0x7f7b0c06ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:40.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 --2- 172.21.15.40:0/737650064 >> v2:172.21.15.53:3300/0 conn(0x7f7b0c145500 0x7f7b0c06ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 >> v1:172.21.15.53:6789/0 conn(0x7f7b0c15bfb0 legacy=0x7f7b0c0716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:40.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c15cd00 con 0x7f7b0c071df0 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f7b040019f0 con 0x7f7b0c071df0 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 >> v1:172.21.15.40:6789/0 conn(0x7f7b0c071df0 legacy=0x7f7b0c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 --2- 172.21.15.40:0/737650064 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7af0003a20 0x7f7af0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 --2- 172.21.15.40:0/737650064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 0x7f7af0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 --2- 172.21.15.40:0/737650064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00052e0 0x7f7af000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c15ca20 con 0x7f7af0004800 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af000c200 con 0x7f7af00052e0 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af000c400 con 0x7f7af0003a20 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.176+0000 7f7b03e00640 1 --2- 172.21.15.40:0/737650064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 0x7f7af0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.177+0000 7f7b03e00640 1 -- 172.21.15.40:0/737650064 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7af0003a20 msgr2=0x7f7af0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.177+0000 7f7b03e00640 1 --2- 172.21.15.40:0/737650064 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7af0003a20 0x7f7af0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.177+0000 7f7b03e00640 1 -- 172.21.15.40:0/737650064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00052e0 msgr2=0x7f7af000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.177+0000 7f7b03e00640 1 --2- 172.21.15.40:0/737650064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00052e0 0x7f7af000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.177+0000 7f7b0be00640 1 -- 172.21.15.40:0/737650064 reap_dead start 2024-03-08T06:04:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.177+0000 7f7b03e00640 1 -- 172.21.15.40:0/737650064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af0003520 con 0x7f7af0004800 2024-03-08T06:04:40.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.178+0000 7f7b03e00640 1 --2- 172.21.15.40:0/737650064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 0x7f7af0004c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7af4003db0 tx=0x7f7af0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd7384e1609442a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:40.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.178+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4037070 con 0x7f7af0004800 2024-03-08T06:04:40.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.178+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4032600 con 0x7f7af0004800 2024-03-08T06:04:40.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.178+0000 7f7b0aa00640 1 -- 172.21.15.40:0/737650064 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4003980 con 0x7f7af0004800 2024-03-08T06:04:40.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.179+0000 7f7b11000640 1 -- 172.21.15.40:0/737650064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 msgr2=0x7f7af0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.179+0000 7f7b11000640 1 --2- 172.21.15.40:0/737650064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 0x7f7af0004c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7af4003db0 tx=0x7f7af0005830 comp rx=0 tx=0).stop 2024-03-08T06:04:40.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.179+0000 7f7b11000640 1 -- 172.21.15.40:0/737650064 shutdown_connections 2024-03-08T06:04:40.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.179+0000 7f7b11000640 1 --2- 172.21.15.40:0/737650064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 0x7f7af0004c80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.179+0000 7f7b11000640 1 -- 172.21.15.40:0/737650064 >> 172.21.15.40:0/737650064 conn(0x7f7b0c0b1850 msgr2=0x7f7b0c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:40.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.179+0000 7f7b11000640 1 -- 172.21.15.40:0/737650064 shutdown_connections 2024-03-08T06:04:40.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.179+0000 7f7b11000640 1 -- 172.21.15.40:0/737650064 wait complete. 2024-03-08T06:04:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.181+0000 7f7b11000640 1 Processor -- start 2024-03-08T06:04:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.181+0000 7f7b11000640 1 -- start start 2024-03-08T06:04:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.182+0000 7f7b11000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 0x7f7b0c134500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.182+0000 7f7b11000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b0c1f4c10 0x7f7b0c1f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.182+0000 7f7b11000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b0c0aaf30 0x7f7b0c0ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.182+0000 7f7b11000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c071a70 con 0x7f7af0004800 2024-03-08T06:04:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.182+0000 7f7b11000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c0718f0 con 0x7f7b0c1f4c10 2024-03-08T06:04:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.182+0000 7f7b11000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c071bf0 con 0x7f7b0c0aaf30 2024-03-08T06:04:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.182+0000 7f7b0b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 0x7f7b0c134500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.182+0000 7f7b0b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 0x7f7b0c134500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45322/0 (socket says 172.21.15.40:45322) 2024-03-08T06:04:40.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.182+0000 7f7b0b400640 1 -- 172.21.15.40:0/2167006247 learned_addr learned my addr 172.21.15.40:0/2167006247 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b03e00640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b0c1f4c10 0x7f7b0c1f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b0be00640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b0c0aaf30 0x7f7b0c0ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b0b400640 1 -- 172.21.15.40:0/2167006247 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b0c0aaf30 msgr2=0x7f7b0c0ab3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b0b400640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b0c0aaf30 0x7f7b0c0ab3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b0b400640 1 -- 172.21.15.40:0/2167006247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b0c1f4c10 msgr2=0x7f7b0c1f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b0b400640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b0c1f4c10 0x7f7b0c1f5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b0b400640 1 -- 172.21.15.40:0/2167006247 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c1e20b0 con 0x7f7af0004800 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b0be00640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b0c0aaf30 0x7f7b0c0ab3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b0b400640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 0x7f7b0c134500 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00d530 tx=0x7f7afc00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b03400640 1 -- 172.21.15.40:0/2167006247 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0125f0 con 0x7f7af0004800 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b03400640 1 -- 172.21.15.40:0/2167006247 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc019070 con 0x7f7af0004800 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.183+0000 7f7b11000640 1 -- 172.21.15.40:0/2167006247 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c1e2390 con 0x7f7af0004800 2024-03-08T06:04:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.184+0000 7f7b03400640 1 -- 172.21.15.40:0/2167006247 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0129d0 con 0x7f7af0004800 2024-03-08T06:04:40.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.184+0000 7f7b11000640 1 -- 172.21.15.40:0/2167006247 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c134a40 con 0x7f7af0004800 2024-03-08T06:04:40.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.185+0000 7f7b11000640 1 -- 172.21.15.40:0/2167006247 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c075090 con 0x7f7af0004800 2024-03-08T06:04:40.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.186+0000 7f7b03400640 1 -- 172.21.15.40:0/2167006247 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7afc012b70 con 0x7f7af0004800 2024-03-08T06:04:40.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.187+0000 7f7b03400640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7ae806b520 0x7f7ae806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.187+0000 7f7b03400640 1 -- 172.21.15.40:0/2167006247 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7afc08a470 con 0x7f7af0004800 2024-03-08T06:04:40.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.187+0000 7f7b03e00640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7ae806b520 0x7f7ae806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.188+0000 7f7b03e00640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7ae806b520 0x7f7ae806d9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7af4034000 tx=0x7f7af4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:40.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.192+0000 7f7b03400640 1 -- 172.21.15.40:0/2167006247 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7afc059360 con 0x7f7af0004800 2024-03-08T06:04:40.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.391+0000 7f7b11000640 1 -- 172.21.15.40:0/2167006247 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b0c0b04d0 con 0x7f7af0004800 2024-03-08T06:04:40.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.393+0000 7f7b03400640 1 -- 172.21.15.40:0/2167006247 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7afc058c60 con 0x7f7af0004800 2024-03-08T06:04:40.394 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.395+0000 7f7b11000640 1 -- 172.21.15.40:0/2167006247 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7ae806b520 msgr2=0x7f7ae806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.395+0000 7f7b11000640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7ae806b520 0x7f7ae806d9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7af4034000 tx=0x7f7af4009030 comp rx=0 tx=0).stop 2024-03-08T06:04:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.395+0000 7f7b11000640 1 -- 172.21.15.40:0/2167006247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 msgr2=0x7f7b0c134500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.395+0000 7f7b11000640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 0x7f7b0c134500 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00d530 tx=0x7f7afc00d9f0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.395+0000 7f7b11000640 1 -- 172.21.15.40:0/2167006247 shutdown_connections 2024-03-08T06:04:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.395+0000 7f7b11000640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7ae806b520 0x7f7ae806d9e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.395+0000 7f7b11000640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b0c0aaf30 0x7f7b0c0ab3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.395+0000 7f7b11000640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b0c1f4c10 0x7f7b0c1f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.395+0000 7f7b11000640 1 --2- 172.21.15.40:0/2167006247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af0004800 0x7f7b0c134500 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.395+0000 7f7b11000640 1 -- 172.21.15.40:0/2167006247 >> 172.21.15.40:0/2167006247 conn(0x7f7b0c0b1850 msgr2=0x7f7b0c06dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:40.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.395+0000 7f7b11000640 1 -- 172.21.15.40:0/2167006247 shutdown_connections 2024-03-08T06:04:40.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.396+0000 7f7b11000640 1 -- 172.21.15.40:0/2167006247 wait complete. 2024-03-08T06:04:40.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:40.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.638+0000 7fd843e00640 1 Processor -- start 2024-03-08T06:04:40.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.638+0000 7fd843e00640 1 -- start start 2024-03-08T06:04:40.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.640+0000 7fd843e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd84415fb30 0x7fd8440b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.640+0000 7fd843e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8440b4e10 con 0x7fd8441667b0 2024-03-08T06:04:40.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.640+0000 7fd843e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd844162730 con 0x7fd844071df0 2024-03-08T06:04:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.640+0000 7fd843e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd844162a10 con 0x7fd84415fb30 2024-03-08T06:04:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.641+0000 7fd842000640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd8441667b0 0x7fd8440716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42642/0 (socket says 172.21.15.40:42642) 2024-03-08T06:04:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.641+0000 7fd842000640 1 -- 172.21.15.40:0/4266767662 learned_addr learned my addr 172.21.15.40:0/4266767662 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.641+0000 7fd842a00640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd84415fb30 0x7fd8440b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.641+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581594256 0 0) 0x7fd8440b4e10 con 0x7fd8441667b0 2024-03-08T06:04:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.641+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd828003520 con 0x7fd8441667b0 2024-03-08T06:04:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4249588400 0 0) 0x7fd828003520 con 0x7fd8441667b0 2024-03-08T06:04:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd84415fb30 msgr2=0x7fd8440b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd84415fb30 0x7fd8440b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 >> v1:172.21.15.53:6789/0 conn(0x7fd844071df0 legacy=0x7fd8440b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd844162cf0 con 0x7fd8441667b0 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fd830001670 con 0x7fd8441667b0 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 >> v1:172.21.15.40:6789/0 conn(0x7fd8441667b0 legacy=0x7fd8440716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd828003a50 0x7fd828003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd84415fb30 0x7fd828004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd828004d40 0x7fd8280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd844162a10 con 0x7fd828003a50 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82800bad0 con 0x7fd828004d40 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82800bcd0 con 0x7fd84415fb30 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd842a00640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd828003a50 0x7fd828003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd842a00640 1 -- 172.21.15.40:0/4266767662 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd84415fb30 msgr2=0x7fd828004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd842a00640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd84415fb30 0x7fd828004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd842a00640 1 -- 172.21.15.40:0/4266767662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd828004d40 msgr2=0x7fd8280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.642+0000 7fd843400640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd828004d40 0x7fd8280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.643+0000 7fd842a00640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd828004d40 0x7fd8280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.643+0000 7fd842a00640 1 -- 172.21.15.40:0/4266767662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd828003520 con 0x7fd828003a50 2024-03-08T06:04:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.643+0000 7fd843400640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd828004d40 0x7fd8280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.644+0000 7fd842a00640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd828003a50 0x7fd828003e70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd834009a90 tx=0x7fd828005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d534867e82050740 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.644+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834039070 con 0x7fd828003a50 2024-03-08T06:04:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.644+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83402c600 con 0x7fd828003a50 2024-03-08T06:04:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.644+0000 7fd841600640 1 -- 172.21.15.40:0/4266767662 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834009690 con 0x7fd828003a50 2024-03-08T06:04:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.644+0000 7fd843e00640 1 -- 172.21.15.40:0/4266767662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd828003a50 msgr2=0x7fd828003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.644+0000 7fd843e00640 1 --2- 172.21.15.40:0/4266767662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd828003a50 0x7fd828003e70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd834009a90 tx=0x7fd828005830 comp rx=0 tx=0).stop 2024-03-08T06:04:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.645+0000 7fd843400640 1 -- 172.21.15.40:0/4266767662 reap_dead start 2024-03-08T06:04:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.645+0000 7fd843e00640 1 -- 172.21.15.40:0/4266767662 shutdown_connections 2024-03-08T06:04:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.645+0000 7fd843e00640 1 -- 172.21.15.40:0/4266767662 >> 172.21.15.40:0/4266767662 conn(0x7fd8440b1850 msgr2=0x7fd8440b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.645+0000 7fd843e00640 1 -- 172.21.15.40:0/4266767662 shutdown_connections 2024-03-08T06:04:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.645+0000 7fd843e00640 1 -- 172.21.15.40:0/4266767662 wait complete. 2024-03-08T06:04:40.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.646+0000 7fd843e00640 1 Processor -- start 2024-03-08T06:04:40.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.647+0000 7fd843e00640 1 -- start start 2024-03-08T06:04:40.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.647+0000 7fd843e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd828003a50 0x7fd8441469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd844154a00 0x7fd8441471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd844147700 0x7fd844164d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84415ff10 con 0x7fd844147700 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8441634f0 con 0x7fd844154a00 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440b4530 con 0x7fd828003a50 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd844147700 0x7fd844164d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd844147700 0x7fd844164d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45342/0 (socket says 172.21.15.40:45342) 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843400640 1 -- 172.21.15.40:0/4052674182 learned_addr learned my addr 172.21.15.40:0/4052674182 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843400640 1 -- 172.21.15.40:0/4052674182 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd828003a50 msgr2=0x7fd8441469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd842000640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd844154a00 0x7fd8441471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd842a00640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd828003a50 0x7fd8441469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843400640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd828003a50 0x7fd8441469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843400640 1 -- 172.21.15.40:0/4052674182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd844154a00 msgr2=0x7fd8441471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843400640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd844154a00 0x7fd8441471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.648+0000 7fd843400640 1 -- 172.21.15.40:0/4052674182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd844146ee0 con 0x7fd844147700 2024-03-08T06:04:40.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.649+0000 7fd843400640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd844147700 0x7fd844164d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd8440b1620 tx=0x7fd83c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:40.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.649+0000 7fd83aa00640 1 -- 172.21.15.40:0/4052674182 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c006d50 con 0x7fd844147700 2024-03-08T06:04:40.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.649+0000 7fd83aa00640 1 -- 172.21.15.40:0/4052674182 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c003970 con 0x7fd844147700 2024-03-08T06:04:40.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.649+0000 7fd83aa00640 1 -- 172.21.15.40:0/4052674182 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c003da0 con 0x7fd844147700 2024-03-08T06:04:40.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.649+0000 7fd843e00640 1 -- 172.21.15.40:0/4052674182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84406db10 con 0x7fd844147700 2024-03-08T06:04:40.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.649+0000 7fd843e00640 1 -- 172.21.15.40:0/4052674182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd844164820 con 0x7fd844147700 2024-03-08T06:04:40.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.650+0000 7fd843e00640 1 -- 172.21.15.40:0/4052674182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd844075090 con 0x7fd844147700 2024-03-08T06:04:40.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.651+0000 7fd83aa00640 1 -- 172.21.15.40:0/4052674182 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd83c01b020 con 0x7fd844147700 2024-03-08T06:04:40.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.652+0000 7fd83aa00640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd82406b4d0 0x7fd82406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:40.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.652+0000 7fd83aa00640 1 -- 172.21.15.40:0/4052674182 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd83c089d20 con 0x7fd844147700 2024-03-08T06:04:40.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.652+0000 7fd842a00640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd82406b4d0 0x7fd82406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:40.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.653+0000 7fd842a00640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd82406b4d0 0x7fd82406d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd834036000 tx=0x7fd834030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:40.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.657+0000 7fd83aa00640 1 -- 172.21.15.40:0/4052674182 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd844146ee0 con 0x7fd844147700 2024-03-08T06:04:40.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.857+0000 7fd843e00640 1 -- 172.21.15.40:0/4052674182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd84406f260 con 0x7fd844147700 2024-03-08T06:04:40.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.858+0000 7fd83aa00640 1 -- 172.21.15.40:0/4052674182 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd83c058ba0 con 0x7fd844147700 2024-03-08T06:04:40.859 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.862+0000 7fd843e00640 1 -- 172.21.15.40:0/4052674182 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd82406b4d0 msgr2=0x7fd82406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.862+0000 7fd843e00640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd82406b4d0 0x7fd82406d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd834036000 tx=0x7fd834030000 comp rx=0 tx=0).stop 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.862+0000 7fd843e00640 1 -- 172.21.15.40:0/4052674182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd844147700 msgr2=0x7fd844164d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.862+0000 7fd843e00640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd844147700 0x7fd844164d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd8440b1620 tx=0x7fd83c00dc00 comp rx=0 tx=0).stop 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.862+0000 7fd843e00640 1 -- 172.21.15.40:0/4052674182 shutdown_connections 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.862+0000 7fd843e00640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fd82406b4d0 0x7fd82406d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.862+0000 7fd843e00640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd844147700 0x7fd844164d40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.862+0000 7fd843e00640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd844154a00 0x7fd8441471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.862+0000 7fd843e00640 1 --2- 172.21.15.40:0/4052674182 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd828003a50 0x7fd8441469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.862+0000 7fd843e00640 1 -- 172.21.15.40:0/4052674182 >> 172.21.15.40:0/4052674182 conn(0x7fd8440b1850 msgr2=0x7fd844165a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.863+0000 7fd843e00640 1 -- 172.21.15.40:0/4052674182 shutdown_connections 2024-03-08T06:04:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:40.863+0000 7fd843e00640 1 -- 172.21.15.40:0/4052674182 wait complete. 2024-03-08T06:04:40.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.106+0000 7f5807e00640 1 Processor -- start 2024-03-08T06:04:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.106+0000 7f5807e00640 1 -- start start 2024-03-08T06:04:41.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.108+0000 7f5807e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58080cf030 0x7f5808161930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.108+0000 7f5807e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f58080b9630 0x7f58080d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.108+0000 7f5807e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580815e240 con 0x7f580815de60 2024-03-08T06:04:41.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.108+0000 7f5807e00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5808162030 con 0x7f58080cf030 2024-03-08T06:04:41.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.108+0000 7f5807e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5808162310 con 0x7f58080b9630 2024-03-08T06:04:41.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.108+0000 7f5806000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58080cf030 0x7f5808161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.108+0000 7f5806000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58080cf030 0x7f5808161930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45350/0 (socket says 172.21.15.40:45350) 2024-03-08T06:04:41.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.108+0000 7f5806000640 1 -- 172.21.15.40:0/3835167761 learned_addr learned my addr 172.21.15.40:0/3835167761 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.108+0000 7f5807400640 1 --2- 172.21.15.40:0/3835167761 >> v2:172.21.15.53:3300/0 conn(0x7f58080b9630 0x7f58080d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.109+0000 7f5806000640 1 -- 172.21.15.40:0/3835167761 >> v2:172.21.15.53:3300/0 conn(0x7f58080b9630 msgr2=0x7f58080d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.109+0000 7f5806000640 1 --2- 172.21.15.40:0/3835167761 >> v2:172.21.15.53:3300/0 conn(0x7f58080b9630 0x7f58080d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.109+0000 7f5806000640 1 -- 172.21.15.40:0/3835167761 >> v1:172.21.15.53:6789/0 conn(0x7f580815de60 legacy=0x7f58080ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:41.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.109+0000 7f5806000640 1 -- 172.21.15.40:0/3835167761 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58081625f0 con 0x7f58080cf030 2024-03-08T06:04:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.110+0000 7f5806000640 1 --2- 172.21.15.40:0/3835167761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58080cf030 0x7f5808161930 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f580808d000 tx=0x7f57f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f960ceb31fc56c53 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.110+0000 7f5805600640 1 -- 172.21.15.40:0/3835167761 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0035020 con 0x7f58080cf030 2024-03-08T06:04:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.110+0000 7f5805600640 1 -- 172.21.15.40:0/3835167761 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f0031870 con 0x7f58080cf030 2024-03-08T06:04:41.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.110+0000 7f5805600640 1 -- 172.21.15.40:0/3835167761 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0031c40 con 0x7f58080cf030 2024-03-08T06:04:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.111+0000 7f5807e00640 1 -- 172.21.15.40:0/3835167761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58080cf030 msgr2=0x7f5808161930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.111+0000 7f5807e00640 1 --2- 172.21.15.40:0/3835167761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58080cf030 0x7f5808161930 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f580808d000 tx=0x7f57f002b910 comp rx=0 tx=0).stop 2024-03-08T06:04:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.111+0000 7f5807e00640 1 -- 172.21.15.40:0/3835167761 shutdown_connections 2024-03-08T06:04:41.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.111+0000 7f5807e00640 1 --2- 172.21.15.40:0/3835167761 >> v2:172.21.15.53:3300/0 conn(0x7f58080b9630 0x7f58080d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.111+0000 7f5807e00640 1 --2- 172.21.15.40:0/3835167761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58080cf030 0x7f5808161930 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.111+0000 7f5807e00640 1 -- 172.21.15.40:0/3835167761 >> 172.21.15.40:0/3835167761 conn(0x7f580806c500 msgr2=0x7f580806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.111+0000 7f5807e00640 1 -- 172.21.15.40:0/3835167761 shutdown_connections 2024-03-08T06:04:41.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.111+0000 7f5807e00640 1 -- 172.21.15.40:0/3835167761 wait complete. 2024-03-08T06:04:41.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.113+0000 7f5807e00640 1 Processor -- start 2024-03-08T06:04:41.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.113+0000 7f5807e00640 1 -- start start 2024-03-08T06:04:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.114+0000 7f5807e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58080b9630 0x7f58080c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.114+0000 7f5807e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58080cf030 0x7f58080c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.114+0000 7f5807e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f580815de60 0x7f5808152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.114+0000 7f5807e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080d2b20 con 0x7f58080b9630 2024-03-08T06:04:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.114+0000 7f5807e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080b9a80 con 0x7f580815de60 2024-03-08T06:04:41.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.114+0000 7f5807e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58080d2ca0 con 0x7f58080cf030 2024-03-08T06:04:41.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.115+0000 7f5806a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58080b9630 0x7f58080c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.115+0000 7f5807400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f580815de60 0x7f5808152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.115+0000 7f5806a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58080b9630 0x7f58080c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45362/0 (socket says 172.21.15.40:45362) 2024-03-08T06:04:41.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.115+0000 7f5806a00640 1 -- 172.21.15.40:0/1455063219 learned_addr learned my addr 172.21.15.40:0/1455063219 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:41.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.115+0000 7f5806000640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58080cf030 0x7f58080c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.115+0000 7f5806a00640 1 -- 172.21.15.40:0/1455063219 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58080cf030 msgr2=0x7f58080c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:41.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.115+0000 7f5806a00640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58080cf030 0x7f58080c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.115+0000 7f5806a00640 1 -- 172.21.15.40:0/1455063219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f580815de60 msgr2=0x7f5808152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:41.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.115+0000 7f5806a00640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f580815de60 0x7f5808152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.115+0000 7f5806a00640 1 -- 172.21.15.40:0/1455063219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58080c9d20 con 0x7f58080b9630 2024-03-08T06:04:41.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.116+0000 7f5806a00640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58080b9630 0x7f58080c8070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f57f800b420 tx=0x7f57f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:41.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.116+0000 7f57fea00640 1 -- 172.21.15.40:0/1455063219 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8011830 con 0x7f58080b9630 2024-03-08T06:04:41.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.116+0000 7f57fea00640 1 -- 172.21.15.40:0/1455063219 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f8016070 con 0x7f58080b9630 2024-03-08T06:04:41.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.116+0000 7f57fea00640 1 -- 172.21.15.40:0/1455063219 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8011c10 con 0x7f58080b9630 2024-03-08T06:04:41.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.116+0000 7f5807e00640 1 -- 172.21.15.40:0/1455063219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58080b4b90 con 0x7f58080b9630 2024-03-08T06:04:41.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.116+0000 7f5807e00640 1 -- 172.21.15.40:0/1455063219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58080b5150 con 0x7f58080b9630 2024-03-08T06:04:41.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.117+0000 7f5807e00640 1 -- 172.21.15.40:0/1455063219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5808075090 con 0x7f58080b9630 2024-03-08T06:04:41.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.119+0000 7f57fea00640 1 -- 172.21.15.40:0/1455063219 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57f801b460 con 0x7f58080b9630 2024-03-08T06:04:41.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.119+0000 7f57fea00640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f57e406b4d0 0x7f57e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.120+0000 7f57fea00640 1 -- 172.21.15.40:0/1455063219 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57f8089c50 con 0x7f58080b9630 2024-03-08T06:04:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.120+0000 7f5806000640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f57e406b4d0 0x7f57e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.120+0000 7f5806000640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f57e406b4d0 0x7f57e406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57f0036000 tx=0x7f57f003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.125+0000 7f57fea00640 1 -- 172.21.15.40:0/1455063219 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57f8020020 con 0x7f58080b9630 2024-03-08T06:04:41.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.322+0000 7f5807e00640 1 -- 172.21.15.40:0/1455063219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58080ca4b0 con 0x7f58080b9630 2024-03-08T06:04:41.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.323+0000 7f57fea00640 1 -- 172.21.15.40:0/1455063219 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57f80583e0 con 0x7f58080b9630 2024-03-08T06:04:41.324 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.325+0000 7f5807e00640 1 -- 172.21.15.40:0/1455063219 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f57e406b4d0 msgr2=0x7f57e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.325+0000 7f5807e00640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f57e406b4d0 0x7f57e406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57f0036000 tx=0x7f57f003c040 comp rx=0 tx=0).stop 2024-03-08T06:04:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.325+0000 7f5807e00640 1 -- 172.21.15.40:0/1455063219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58080b9630 msgr2=0x7f58080c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.325+0000 7f5807e00640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58080b9630 0x7f58080c8070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f57f800b420 tx=0x7f57f800b8e0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.326+0000 7f5807e00640 1 -- 172.21.15.40:0/1455063219 shutdown_connections 2024-03-08T06:04:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.326+0000 7f5807e00640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f57e406b4d0 0x7f57e406d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.326+0000 7f5807e00640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f580815de60 0x7f5808152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.326+0000 7f5807e00640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58080cf030 0x7f58080c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.326+0000 7f5807e00640 1 --2- 172.21.15.40:0/1455063219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58080b9630 0x7f58080c8070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.326+0000 7f5807e00640 1 -- 172.21.15.40:0/1455063219 >> 172.21.15.40:0/1455063219 conn(0x7f580806c500 msgr2=0x7f58080ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:41.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.326+0000 7f5807e00640 1 -- 172.21.15.40:0/1455063219 shutdown_connections 2024-03-08T06:04:41.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.326+0000 7f5807e00640 1 -- 172.21.15.40:0/1455063219 wait complete. 2024-03-08T06:04:41.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters' 2024-03-08T06:04:41.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.573+0000 7f48e2000640 1 Processor -- start 2024-03-08T06:04:41.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.574+0000 7f48e2000640 1 -- start start 2024-03-08T06:04:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.575+0000 7f48e2000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48dc071ab0 0x7f48dc15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.575+0000 7f48e2000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f48dc071e90 0x7f48dc15d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.575+0000 7f48e2000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48dc15d0c0 con 0x7f48dc06d600 2024-03-08T06:04:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.575+0000 7f48e2000640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48dc06ef40 con 0x7f48dc071ab0 2024-03-08T06:04:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.575+0000 7f48e2000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc06da80 con 0x7f48dc071e90 2024-03-08T06:04:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.575+0000 7f48db400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48dc071ab0 0x7f48dc15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.575+0000 7f48db400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48dc071ab0 0x7f48dc15cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45374/0 (socket says 172.21.15.40:45374) 2024-03-08T06:04:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.575+0000 7f48db400640 1 -- 172.21.15.40:0/191137695 learned_addr learned my addr 172.21.15.40:0/191137695 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.576+0000 7f48dbe00640 1 --2- 172.21.15.40:0/191137695 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f48dc071e90 0x7f48dc15d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.576+0000 7f48db400640 1 -- 172.21.15.40:0/191137695 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f48dc071e90 msgr2=0x7f48dc15d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:41.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.576+0000 7f48db400640 1 --2- 172.21.15.40:0/191137695 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f48dc071e90 0x7f48dc15d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.576+0000 7f48db400640 1 -- 172.21.15.40:0/191137695 >> v1:172.21.15.53:6789/0 conn(0x7f48dc06d600 legacy=0x7f48dc144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:41.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.576+0000 7f48db400640 1 -- 172.21.15.40:0/191137695 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc06dd60 con 0x7f48dc071ab0 2024-03-08T06:04:41.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.577+0000 7f48db400640 1 --2- 172.21.15.40:0/191137695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48dc071ab0 0x7f48dc15cb80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009870 tx=0x7f48cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b35a6cab21bbc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:41.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.577+0000 7f48daa00640 1 -- 172.21.15.40:0/191137695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc035020 con 0x7f48dc071ab0 2024-03-08T06:04:41.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.577+0000 7f48daa00640 1 -- 172.21.15.40:0/191137695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc031640 con 0x7f48dc071ab0 2024-03-08T06:04:41.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.577+0000 7f48daa00640 1 -- 172.21.15.40:0/191137695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc02d040 con 0x7f48dc071ab0 2024-03-08T06:04:41.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.578+0000 7f48e2000640 1 -- 172.21.15.40:0/191137695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48dc071ab0 msgr2=0x7f48dc15cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:41.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.578+0000 7f48e2000640 1 --2- 172.21.15.40:0/191137695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48dc071ab0 0x7f48dc15cb80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009870 tx=0x7f48cc02b8f0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.578+0000 7f48e2000640 1 -- 172.21.15.40:0/191137695 shutdown_connections 2024-03-08T06:04:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.578+0000 7f48e2000640 1 --2- 172.21.15.40:0/191137695 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f48dc071e90 0x7f48dc15d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.578+0000 7f48e2000640 1 --2- 172.21.15.40:0/191137695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48dc071ab0 0x7f48dc15cb80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.578+0000 7f48e2000640 1 -- 172.21.15.40:0/191137695 >> 172.21.15.40:0/191137695 conn(0x7f48dc06ae30 msgr2=0x7f48dc06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.578+0000 7f48e2000640 1 -- 172.21.15.40:0/191137695 shutdown_connections 2024-03-08T06:04:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.578+0000 7f48e2000640 1 -- 172.21.15.40:0/191137695 wait complete. 2024-03-08T06:04:41.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.580+0000 7f48e2000640 1 Processor -- start 2024-03-08T06:04:41.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.580+0000 7f48e2000640 1 -- start start 2024-03-08T06:04:41.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.581+0000 7f48e2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc06d600 0x7f48dc1f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.581+0000 7f48e2000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48dc071ab0 0x7f48dc1e7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.581+0000 7f48e2000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f48dc071e90 0x7f48dc15e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.581+0000 7f48e2000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc06e600 con 0x7f48dc071ab0 2024-03-08T06:04:41.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.581+0000 7f48e2000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc06e320 con 0x7f48dc06d600 2024-03-08T06:04:41.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.581+0000 7f48e2000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc145470 con 0x7f48dc071e90 2024-03-08T06:04:41.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.581+0000 7f48d3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48dc071ab0 0x7f48dc1e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.581+0000 7f48d3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48dc071ab0 0x7f48dc1e7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45384/0 (socket says 172.21.15.40:45384) 2024-03-08T06:04:41.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.581+0000 7f48d3e00640 1 -- 172.21.15.40:0/220933872 learned_addr learned my addr 172.21.15.40:0/220933872 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:41.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48db400640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc06d600 0x7f48dc1f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48dbe00640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f48dc071e90 0x7f48dc15e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48d3e00640 1 -- 172.21.15.40:0/220933872 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f48dc071e90 msgr2=0x7f48dc15e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:41.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48d3e00640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f48dc071e90 0x7f48dc15e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48d3e00640 1 -- 172.21.15.40:0/220933872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc06d600 msgr2=0x7f48dc1f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:41.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48d3e00640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc06d600 0x7f48dc1f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:41.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48d3e00640 1 -- 172.21.15.40:0/220933872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc1e7cb0 con 0x7f48dc071ab0 2024-03-08T06:04:41.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48d3e00640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48dc071ab0 0x7f48dc1e7360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c420 tx=0x7f48c400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:41.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48d3400640 1 -- 172.21.15.40:0/220933872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4007020 con 0x7f48dc071ab0 2024-03-08T06:04:41.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48d3400640 1 -- 172.21.15.40:0/220933872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c40071c0 con 0x7f48dc071ab0 2024-03-08T06:04:41.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48d3400640 1 -- 172.21.15.40:0/220933872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4007590 con 0x7f48dc071ab0 2024-03-08T06:04:41.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.582+0000 7f48e2000640 1 -- 172.21.15.40:0/220933872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48dc1e7f90 con 0x7f48dc071ab0 2024-03-08T06:04:41.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.583+0000 7f48e2000640 1 -- 172.21.15.40:0/220933872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48dc1e8550 con 0x7f48dc071ab0 2024-03-08T06:04:41.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.584+0000 7f48e2000640 1 -- 172.21.15.40:0/220933872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48dc072580 con 0x7f48dc071ab0 2024-03-08T06:04:41.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.585+0000 7f48d3400640 1 -- 172.21.15.40:0/220933872 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48c4019020 con 0x7f48dc071ab0 2024-03-08T06:04:41.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.586+0000 7f48d3400640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f48b806b4d0 0x7f48b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:41.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.586+0000 7f48d3400640 1 -- 172.21.15.40:0/220933872 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48c4089a00 con 0x7f48dc071ab0 2024-03-08T06:04:41.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.586+0000 7f48db400640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f48b806b4d0 0x7f48b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:41.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.587+0000 7f48db400640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f48b806b4d0 0x7f48b806d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f48cc036000 tx=0x7f48cc03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:41.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.591+0000 7f48d3400640 1 -- 172.21.15.40:0/220933872 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48dc072580 con 0x7f48dc071ab0 2024-03-08T06:04:41.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:41.866+0000 7f48e2000640 1 -- 172.21.15.40:0/220933872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]} v 0) v1 -- 0x7f48dc067cb0 con 0x7f48dc071ab0 2024-03-08T06:04:42.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.756+0000 7f48d3400640 1 -- 172.21.15.40:0/220933872 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]}]=0 v56) v1 ==== 167+0+0 (secure 0 0 0) 0x7f48c40587e0 con 0x7f48dc071ab0 2024-03-08T06:04:42.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 -- 172.21.15.40:0/220933872 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f48b806b4d0 msgr2=0x7f48b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:42.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f48b806b4d0 0x7f48b806d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f48cc036000 tx=0x7f48cc03c040 comp rx=0 tx=0).stop 2024-03-08T06:04:42.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 -- 172.21.15.40:0/220933872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48dc071ab0 msgr2=0x7f48dc1e7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:42.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48dc071ab0 0x7f48dc1e7360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c420 tx=0x7f48c400c8e0 comp rx=0 tx=0).stop 2024-03-08T06:04:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 -- 172.21.15.40:0/220933872 shutdown_connections 2024-03-08T06:04:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f48b806b4d0 0x7f48b806d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f48dc071e90 0x7f48dc15e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48dc071ab0 0x7f48dc1e7360 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 --2- 172.21.15.40:0/220933872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48dc06d600 0x7f48dc1f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 -- 172.21.15.40:0/220933872 >> 172.21.15.40:0/220933872 conn(0x7f48dc06ae30 msgr2=0x7f48dc15f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 -- 172.21.15.40:0/220933872 shutdown_connections 2024-03-08T06:04:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.760+0000 7f48e2000640 1 -- 172.21.15.40:0/220933872 wait complete. 2024-03-08T06:04:42.783 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-08T06:04:43.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:42.999+0000 7faa05400640 1 Processor -- start 2024-03-08T06:04:43.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.000+0000 7faa05400640 1 -- start start 2024-03-08T06:04:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.001+0000 7faa05400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa00161910 0x7faa000b9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.001+0000 7faa05400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0015e220 con 0x7faa000cf010 2024-03-08T06:04:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.001+0000 7faa05400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa00161d60 con 0x7faa0015de40 2024-03-08T06:04:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.001+0000 7faa05400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa000b9b50 con 0x7faa00161910 2024-03-08T06:04:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.001+0000 7faa04a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa00161910 0x7faa000b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.002+0000 7fa9ff400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faa000cf010 0x7faa000d25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42656/0 (socket says 172.21.15.40:42656) 2024-03-08T06:04:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.002+0000 7fa9ff400640 1 -- 172.21.15.40:0/194668807 learned_addr learned my addr 172.21.15.40:0/194668807 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.002+0000 7faa04a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa00161910 0x7faa000b9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45390/0 (socket says 172.21.15.40:45390) 2024-03-08T06:04:43.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.002+0000 7faa04a00640 1 -- 172.21.15.40:0/194668807 >> v1:172.21.15.53:6789/0 conn(0x7faa0015de40 legacy=0x7faa000ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.002+0000 7faa04a00640 1 -- 172.21.15.40:0/194668807 >> v1:172.21.15.40:6789/0 conn(0x7faa000cf010 legacy=0x7faa000d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.002+0000 7faa04a00640 1 -- 172.21.15.40:0/194668807 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00162010 con 0x7faa00161910 2024-03-08T06:04:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.003+0000 7faa04a00640 1 --2- 172.21.15.40:0/194668807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa00161910 0x7faa000b9610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80097a0 tx=0x7fa9f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d2315da006cb23 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.003+0000 7fa9fea00640 1 -- 172.21.15.40:0/194668807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8035020 con 0x7faa00161910 2024-03-08T06:04:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.004+0000 7fa9fea00640 1 -- 172.21.15.40:0/194668807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f8031830 con 0x7faa00161910 2024-03-08T06:04:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.004+0000 7fa9fea00640 1 -- 172.21.15.40:0/194668807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8031c00 con 0x7faa00161910 2024-03-08T06:04:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.004+0000 7faa05400640 1 -- 172.21.15.40:0/194668807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa00161910 msgr2=0x7faa000b9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.004+0000 7faa05400640 1 --2- 172.21.15.40:0/194668807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa00161910 0x7faa000b9610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80097a0 tx=0x7fa9f802b8d0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.004+0000 7faa05400640 1 -- 172.21.15.40:0/194668807 shutdown_connections 2024-03-08T06:04:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.004+0000 7faa05400640 1 --2- 172.21.15.40:0/194668807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa00161910 0x7faa000b9610 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.004+0000 7faa05400640 1 -- 172.21.15.40:0/194668807 >> 172.21.15.40:0/194668807 conn(0x7faa0006c500 msgr2=0x7faa0006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.004+0000 7faa05400640 1 -- 172.21.15.40:0/194668807 shutdown_connections 2024-03-08T06:04:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.004+0000 7faa05400640 1 -- 172.21.15.40:0/194668807 wait complete. 2024-03-08T06:04:43.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.006+0000 7faa05400640 1 Processor -- start 2024-03-08T06:04:43.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.006+0000 7faa05400640 1 -- start start 2024-03-08T06:04:43.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.007+0000 7faa05400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa000cf010 0x7faa001e89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:43.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.007+0000 7faa05400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa0015de40 0x7faa000c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:43.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.007+0000 7faa05400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa00161910 0x7faa000c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:43.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.007+0000 7faa05400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa000d20a0 con 0x7faa00161910 2024-03-08T06:04:43.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.007+0000 7faa05400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa001625d0 con 0x7faa000cf010 2024-03-08T06:04:43.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.007+0000 7faa05400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa000d2cd0 con 0x7faa0015de40 2024-03-08T06:04:43.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.007+0000 7faa04a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa00161910 0x7faa000c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.007+0000 7faa04a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa00161910 0x7faa000c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45392/0 (socket says 172.21.15.40:45392) 2024-03-08T06:04:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.007+0000 7faa04a00640 1 -- 172.21.15.40:0/3938720895 learned_addr learned my addr 172.21.15.40:0/3938720895 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.008+0000 7fa9ff400640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa0015de40 0x7faa000c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.008+0000 7fa9ffe00640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa000cf010 0x7faa001e89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.008+0000 7faa04a00640 1 -- 172.21.15.40:0/3938720895 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa0015de40 msgr2=0x7faa000c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.008+0000 7faa04a00640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa0015de40 0x7faa000c7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.008+0000 7faa04a00640 1 -- 172.21.15.40:0/3938720895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa000cf010 msgr2=0x7faa001e89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.008+0000 7faa04a00640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa000cf010 0x7faa001e89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.008+0000 7faa04a00640 1 -- 172.21.15.40:0/3938720895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa000c8200 con 0x7faa00161910 2024-03-08T06:04:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.008+0000 7fa9ffe00640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa000cf010 0x7faa001e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.008+0000 7faa04a00640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa00161910 0x7faa000c7b80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80097a0 tx=0x7fa9f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:43.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.009+0000 7fa9f7e00640 1 -- 172.21.15.40:0/3938720895 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8040070 con 0x7faa00161910 2024-03-08T06:04:43.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.009+0000 7fa9f7e00640 1 -- 172.21.15.40:0/3938720895 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f80398c0 con 0x7faa00161910 2024-03-08T06:04:43.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.009+0000 7fa9f7e00640 1 -- 172.21.15.40:0/3938720895 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8035040 con 0x7faa00161910 2024-03-08T06:04:43.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.009+0000 7faa05400640 1 -- 172.21.15.40:0/3938720895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa000c84e0 con 0x7faa00161910 2024-03-08T06:04:43.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.009+0000 7faa05400640 1 -- 172.21.15.40:0/3938720895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa000c8aa0 con 0x7faa00161910 2024-03-08T06:04:43.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.010+0000 7faa05400640 1 -- 172.21.15.40:0/3938720895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa000b6820 con 0x7faa00161910 2024-03-08T06:04:43.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.011+0000 7fa9f7e00640 1 -- 172.21.15.40:0/3938720895 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9f804b060 con 0x7faa00161910 2024-03-08T06:04:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.012+0000 7fa9f7e00640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9dc06b4d0 0x7fa9dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.012+0000 7fa9f7e00640 1 -- 172.21.15.40:0/3938720895 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9f80abb90 con 0x7faa00161910 2024-03-08T06:04:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.012+0000 7fa9ffe00640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9dc06b4d0 0x7fa9dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.013+0000 7fa9ffe00640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9dc06b4d0 0x7fa9dc06d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0004520 tx=0x7fa9f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:43.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.017+0000 7fa9f7e00640 1 -- 172.21.15.40:0/3938720895 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa000b6820 con 0x7faa00161910 2024-03-08T06:04:43.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.216+0000 7faa05400640 1 -- 172.21.15.40:0/3938720895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faa00159e50 con 0x7faa00161910 2024-03-08T06:04:43.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.224+0000 7fa9f7e00640 1 -- 172.21.15.40:0/3938720895 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa00159e50 con 0x7faa00161910 2024-03-08T06:04:43.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.226+0000 7faa05400640 1 -- 172.21.15.40:0/3938720895 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9dc06b4d0 msgr2=0x7fa9dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:43.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.226+0000 7faa05400640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9dc06b4d0 0x7fa9dc06d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0004520 tx=0x7fa9f0009040 comp rx=0 tx=0).stop 2024-03-08T06:04:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.226+0000 7faa05400640 1 -- 172.21.15.40:0/3938720895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa00161910 msgr2=0x7faa000c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.226+0000 7faa05400640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa00161910 0x7faa000c7b80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80097a0 tx=0x7fa9f80046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.227+0000 7faa05400640 1 -- 172.21.15.40:0/3938720895 shutdown_connections 2024-03-08T06:04:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.227+0000 7faa05400640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9dc06b4d0 0x7fa9dc06d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.227+0000 7faa05400640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa00161910 0x7faa000c7b80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.227+0000 7faa05400640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa0015de40 0x7faa000c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.227+0000 7faa05400640 1 --2- 172.21.15.40:0/3938720895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa000cf010 0x7faa001e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.227+0000 7faa05400640 1 -- 172.21.15.40:0/3938720895 >> 172.21.15.40:0/3938720895 conn(0x7faa0006c500 msgr2=0x7faa000ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.227+0000 7faa05400640 1 -- 172.21.15.40:0/3938720895 shutdown_connections 2024-03-08T06:04:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.227+0000 7faa05400640 1 -- 172.21.15.40:0/3938720895 wait complete. 2024-03-08T06:04:43.249 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-08T06:04:43.249 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T06:04:43.367 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T06:04:43.495 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-08T06:04:43.495 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:04:43.495 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:43.557 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:04:43.558 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:43.614 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:04:43.614 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:43.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:43.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.895+0000 7f7f9a000640 1 Processor -- start 2024-03-08T06:04:43.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.895+0000 7f7f9a000640 1 -- start start 2024-03-08T06:04:43.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.897+0000 7f7f9a000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f94145520 0x7f7f9415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:43.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.897+0000 7f7f9a000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9415ceb0 con 0x7f7f94071df0 2024-03-08T06:04:43.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.897+0000 7f7f9a000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9415c330 con 0x7f7f9415bea0 2024-03-08T06:04:43.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.897+0000 7f7f9a000640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9415c610 con 0x7f7f94145520 2024-03-08T06:04:43.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.897+0000 7f7f98c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f94145520 0x7f7f9415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:43.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.898+0000 7f7f99600640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7f94071df0 0x7f7f9406d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42672/0 (socket says 172.21.15.40:42672) 2024-03-08T06:04:43.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.898+0000 7f7f99600640 1 -- 172.21.15.40:0/396855418 learned_addr learned my addr 172.21.15.40:0/396855418 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:43.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.898+0000 7f7f98c00640 1 -- 172.21.15.40:0/396855418 >> v1:172.21.15.53:6789/0 conn(0x7f7f9415bea0 legacy=0x7f7f940716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:43.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.898+0000 7f7f98c00640 1 -- 172.21.15.40:0/396855418 >> v1:172.21.15.40:6789/0 conn(0x7f7f94071df0 legacy=0x7f7f9406d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:43.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.898+0000 7f7f98c00640 1 -- 172.21.15.40:0/396855418 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9415c8f0 con 0x7f7f94145520 2024-03-08T06:04:43.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.898+0000 7f7f93400640 1 -- 172.21.15.40:0/396855418 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341248878 0 0) 0x7f7f9415ceb0 con 0x7f7f94071df0 2024-03-08T06:04:43.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.899+0000 7f7f98c00640 1 --2- 172.21.15.40:0/396855418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f94145520 0x7f7f9415d0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7f880097a0 tx=0x7f7f8802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3017145fb7f422 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:43.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.900+0000 7f7f93400640 1 -- 172.21.15.40:0/396855418 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88034070 con 0x7f7f94145520 2024-03-08T06:04:43.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.900+0000 7f7f93400640 1 -- 172.21.15.40:0/396855418 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f88030610 con 0x7f7f94145520 2024-03-08T06:04:43.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.900+0000 7f7f93400640 1 -- 172.21.15.40:0/396855418 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880309e0 con 0x7f7f94145520 2024-03-08T06:04:43.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.900+0000 7f7f9a000640 1 -- 172.21.15.40:0/396855418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f94145520 msgr2=0x7f7f9415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:43.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.900+0000 7f7f9a000640 1 --2- 172.21.15.40:0/396855418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f94145520 0x7f7f9415d0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7f880097a0 tx=0x7f7f8802b960 comp rx=0 tx=0).stop 2024-03-08T06:04:43.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.901+0000 7f7f9a000640 1 -- 172.21.15.40:0/396855418 shutdown_connections 2024-03-08T06:04:43.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.901+0000 7f7f9a000640 1 --2- 172.21.15.40:0/396855418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f94145520 0x7f7f9415d0a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.901+0000 7f7f9a000640 1 -- 172.21.15.40:0/396855418 >> 172.21.15.40:0/396855418 conn(0x7f7f940b1850 msgr2=0x7f7f940b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:43.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.901+0000 7f7f9a000640 1 -- 172.21.15.40:0/396855418 shutdown_connections 2024-03-08T06:04:43.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.901+0000 7f7f9a000640 1 -- 172.21.15.40:0/396855418 wait complete. 2024-03-08T06:04:43.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.902+0000 7f7f9a000640 1 Processor -- start 2024-03-08T06:04:43.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.903+0000 7f7f9a000640 1 -- start start 2024-03-08T06:04:43.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.903+0000 7f7f9a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f94071df0 0x7f7f94143290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:43.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.903+0000 7f7f9a000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f94145520 0x7f7f9415f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:43.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f9a000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f9415f7b0 0x7f7f9413ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f9a000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940af1c0 con 0x7f7f9415f7b0 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f9a000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940af040 con 0x7f7f94071df0 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f9a000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9406e410 con 0x7f7f94145520 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f99600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f9415f7b0 0x7f7f9413ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f99600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f9415f7b0 0x7f7f9413ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45412/0 (socket says 172.21.15.40:45412) 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f99600640 1 -- 172.21.15.40:0/964440623 learned_addr learned my addr 172.21.15.40:0/964440623 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f98c00640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f94071df0 0x7f7f94143290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f93e00640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f94145520 0x7f7f9415f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f99600640 1 -- 172.21.15.40:0/964440623 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f94145520 msgr2=0x7f7f9415f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f99600640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f94145520 0x7f7f9415f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f99600640 1 -- 172.21.15.40:0/964440623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f94071df0 msgr2=0x7f7f94143290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:43.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f99600640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f94071df0 0x7f7f94143290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:43.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.904+0000 7f7f99600640 1 -- 172.21.15.40:0/964440623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9413b710 con 0x7f7f9415f7b0 2024-03-08T06:04:43.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.905+0000 7f7f99600640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f9415f7b0 0x7f7f9413ab30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c760 tx=0x7f7f8c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:43.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.905+0000 7f7f90c00640 1 -- 172.21.15.40:0/964440623 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c018070 con 0x7f7f9415f7b0 2024-03-08T06:04:43.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.905+0000 7f7f90c00640 1 -- 172.21.15.40:0/964440623 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c00a640 con 0x7f7f9415f7b0 2024-03-08T06:04:43.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.905+0000 7f7f90c00640 1 -- 172.21.15.40:0/964440623 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00aa30 con 0x7f7f9415f7b0 2024-03-08T06:04:43.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.905+0000 7f7f9a000640 1 -- 172.21.15.40:0/964440623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9413b9f0 con 0x7f7f9415f7b0 2024-03-08T06:04:43.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.905+0000 7f7f9a000640 1 -- 172.21.15.40:0/964440623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9413bfb0 con 0x7f7f9415f7b0 2024-03-08T06:04:43.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.906+0000 7f7f9a000640 1 -- 172.21.15.40:0/964440623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f94075100 con 0x7f7f9415f7b0 2024-03-08T06:04:43.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.907+0000 7f7f90c00640 1 -- 172.21.15.40:0/964440623 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f8c00abd0 con 0x7f7f9415f7b0 2024-03-08T06:04:43.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.908+0000 7f7f90c00640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7f7406b4d0 0x7f7f7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:43.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.908+0000 7f7f90c00640 1 -- 172.21.15.40:0/964440623 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f8c0898a0 con 0x7f7f9415f7b0 2024-03-08T06:04:43.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.909+0000 7f7f98c00640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7f7406b4d0 0x7f7f7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:43.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.910+0000 7f7f98c00640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7f7406b4d0 0x7f7f7406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7f88036000 tx=0x7f7f8803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:43.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:43.913+0000 7f7f90c00640 1 -- 172.21.15.40:0/964440623 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f8c058680 con 0x7f7f9415f7b0 2024-03-08T06:04:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.113+0000 7f7f9a000640 1 -- 172.21.15.40:0/964440623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f940ceea0 con 0x7f7f9415f7b0 2024-03-08T06:04:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.116+0000 7f7f90c00640 1 -- 172.21.15.40:0/964440623 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f7f8c057f80 con 0x7f7f9415f7b0 2024-03-08T06:04:44.117 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 -- 172.21.15.40:0/964440623 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7f7406b4d0 msgr2=0x7f7f7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7f7406b4d0 0x7f7f7406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7f88036000 tx=0x7f7f8803b040 comp rx=0 tx=0).stop 2024-03-08T06:04:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 -- 172.21.15.40:0/964440623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f9415f7b0 msgr2=0x7f7f9413ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f9415f7b0 0x7f7f9413ab30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c760 tx=0x7f7f8c00cc20 comp rx=0 tx=0).stop 2024-03-08T06:04:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 -- 172.21.15.40:0/964440623 shutdown_connections 2024-03-08T06:04:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7f7406b4d0 0x7f7f7406d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f9415f7b0 0x7f7f9413ab30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f94145520 0x7f7f9415f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 --2- 172.21.15.40:0/964440623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f94071df0 0x7f7f94143290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 -- 172.21.15.40:0/964440623 >> 172.21.15.40:0/964440623 conn(0x7f7f940b1850 msgr2=0x7f7f940cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 -- 172.21.15.40:0/964440623 shutdown_connections 2024-03-08T06:04:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.119+0000 7f7f9a000640 1 -- 172.21.15.40:0/964440623 wait complete. 2024-03-08T06:04:44.141 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:44.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.370+0000 7f7192800640 1 Processor -- start 2024-03-08T06:04:44.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.370+0000 7f7192800640 1 -- start start 2024-03-08T06:04:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.372+0000 7f7192800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f718c15bfb0 0x7f718c0716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.372+0000 7f7192800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f718c071c20 0x7f718c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.372+0000 7f7192800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718c15c460 con 0x7f718c145500 2024-03-08T06:04:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.372+0000 7f7192800640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718c072050 con 0x7f718c15bfb0 2024-03-08T06:04:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.372+0000 7f7192800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718c15c740 con 0x7f718c071c20 2024-03-08T06:04:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.372+0000 7f7190a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f718c15bfb0 0x7f718c0716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.372+0000 7f7190a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f718c15bfb0 0x7f718c0716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45420/0 (socket says 172.21.15.40:45420) 2024-03-08T06:04:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.372+0000 7f7190a00640 1 -- 172.21.15.40:0/1831125698 learned_addr learned my addr 172.21.15.40:0/1831125698 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.372+0000 7f7191e00640 1 --2- 172.21.15.40:0/1831125698 >> v2:172.21.15.53:3300/0 conn(0x7f718c071c20 0x7f718c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.373+0000 7f7190a00640 1 -- 172.21.15.40:0/1831125698 >> v2:172.21.15.53:3300/0 conn(0x7f718c071c20 msgr2=0x7f718c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.373+0000 7f7190a00640 1 --2- 172.21.15.40:0/1831125698 >> v2:172.21.15.53:3300/0 conn(0x7f718c071c20 0x7f718c0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.373+0000 7f7190a00640 1 -- 172.21.15.40:0/1831125698 >> v1:172.21.15.40:6789/0 conn(0x7f718c145500 legacy=0x7f718c06ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.373+0000 7f7190a00640 1 -- 172.21.15.40:0/1831125698 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c15ca20 con 0x7f718c15bfb0 2024-03-08T06:04:44.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.374+0000 7f7190a00640 1 --2- 172.21.15.40:0/1831125698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f718c15bfb0 0x7f718c0716e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f718c0cbb30 tx=0x7f717802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587886073f6df604 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:44.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.374+0000 7f718be00640 1 -- 172.21.15.40:0/1831125698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178035020 con 0x7f718c15bfb0 2024-03-08T06:04:44.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.374+0000 7f718be00640 1 -- 172.21.15.40:0/1831125698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7178031870 con 0x7f718c15bfb0 2024-03-08T06:04:44.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.374+0000 7f718be00640 1 -- 172.21.15.40:0/1831125698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178031c40 con 0x7f718c15bfb0 2024-03-08T06:04:44.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.375+0000 7f7192800640 1 -- 172.21.15.40:0/1831125698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f718c15bfb0 msgr2=0x7f718c0716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.375+0000 7f7192800640 1 --2- 172.21.15.40:0/1831125698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f718c15bfb0 0x7f718c0716e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f718c0cbb30 tx=0x7f717802b910 comp rx=0 tx=0).stop 2024-03-08T06:04:44.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.375+0000 7f7192800640 1 -- 172.21.15.40:0/1831125698 shutdown_connections 2024-03-08T06:04:44.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.375+0000 7f7192800640 1 --2- 172.21.15.40:0/1831125698 >> v2:172.21.15.53:3300/0 conn(0x7f718c071c20 0x7f718c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.375+0000 7f7192800640 1 --2- 172.21.15.40:0/1831125698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f718c15bfb0 0x7f718c0716e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.375+0000 7f7192800640 1 -- 172.21.15.40:0/1831125698 >> 172.21.15.40:0/1831125698 conn(0x7f718c0b1850 msgr2=0x7f718c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:44.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.375+0000 7f7192800640 1 -- 172.21.15.40:0/1831125698 shutdown_connections 2024-03-08T06:04:44.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.375+0000 7f7192800640 1 -- 172.21.15.40:0/1831125698 wait complete. 2024-03-08T06:04:44.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.377+0000 7f7192800640 1 Processor -- start 2024-03-08T06:04:44.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.377+0000 7f7192800640 1 -- start start 2024-03-08T06:04:44.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7192800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f718c071c20 0x7f718c1f1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7192800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f718c145500 0x7f718c13da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7192800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f718c15bfb0 0x7f718c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7192800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c0ccbc0 con 0x7f718c145500 2024-03-08T06:04:44.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7192800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c0cca40 con 0x7f718c15bfb0 2024-03-08T06:04:44.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7192800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718c0ccd40 con 0x7f718c071c20 2024-03-08T06:04:44.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7190a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f718c145500 0x7f718c13da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:44.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7190a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f718c145500 0x7f718c13da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45436/0 (socket says 172.21.15.40:45436) 2024-03-08T06:04:44.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7190a00640 1 -- 172.21.15.40:0/322590067 learned_addr learned my addr 172.21.15.40:0/322590067 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:44.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7190a00640 1 -- 172.21.15.40:0/322590067 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f718c071c20 msgr2=0x7f718c1f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7190a00640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f718c071c20 0x7f718c1f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7190a00640 1 -- 172.21.15.40:0/322590067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f718c15bfb0 msgr2=0x7f718c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7190a00640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f718c15bfb0 0x7f718c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.378+0000 7f7190a00640 1 -- 172.21.15.40:0/322590067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c155cc0 con 0x7f718c145500 2024-03-08T06:04:44.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.379+0000 7f7190a00640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f718c145500 0x7f718c13da10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7178000c00 tx=0x7f71780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:44.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.379+0000 7f7189600640 1 -- 172.21.15.40:0/322590067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178035040 con 0x7f718c145500 2024-03-08T06:04:44.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.379+0000 7f7189600640 1 -- 172.21.15.40:0/322590067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7178030d00 con 0x7f718c145500 2024-03-08T06:04:44.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.379+0000 7f7192800640 1 -- 172.21.15.40:0/322590067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718c155fa0 con 0x7f718c145500 2024-03-08T06:04:44.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.379+0000 7f7189600640 1 -- 172.21.15.40:0/322590067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71780456a0 con 0x7f718c145500 2024-03-08T06:04:44.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.379+0000 7f7192800640 1 -- 172.21.15.40:0/322590067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c156560 con 0x7f718c145500 2024-03-08T06:04:44.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.380+0000 7f7192800640 1 -- 172.21.15.40:0/322590067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718c075090 con 0x7f718c145500 2024-03-08T06:04:44.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.381+0000 7f7189600640 1 -- 172.21.15.40:0/322590067 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7178038020 con 0x7f718c145500 2024-03-08T06:04:44.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.382+0000 7f7189600640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f716c06b520 0x7f716c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.382+0000 7f7189600640 1 -- 172.21.15.40:0/322590067 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71780ac4a0 con 0x7f718c145500 2024-03-08T06:04:44.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.383+0000 7f7191400640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f716c06b520 0x7f716c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:44.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.383+0000 7f7191400640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f716c06b520 0x7f716c06d9e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7180000f60 tx=0x7f71800079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:44.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.387+0000 7f7189600640 1 -- 172.21.15.40:0/322590067 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f717807b390 con 0x7f718c145500 2024-03-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.591+0000 7f7192800640 1 -- 172.21.15.40:0/322590067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f718c140240 con 0x7f718c145500 2024-03-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.592+0000 7f7189600640 1 -- 172.21.15.40:0/322590067 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f717807ac90 con 0x7f718c145500 2024-03-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 -- 172.21.15.40:0/322590067 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f716c06b520 msgr2=0x7f716c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f716c06b520 0x7f716c06d9e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7180000f60 tx=0x7f71800079d0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 -- 172.21.15.40:0/322590067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f718c145500 msgr2=0x7f718c13da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f718c145500 0x7f718c13da10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7178000c00 tx=0x7f71780046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 -- 172.21.15.40:0/322590067 shutdown_connections 2024-03-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f716c06b520 0x7f716c06d9e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f718c15bfb0 0x7f718c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f718c145500 0x7f718c13da10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 --2- 172.21.15.40:0/322590067 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f718c071c20 0x7f718c1f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 -- 172.21.15.40:0/322590067 >> 172.21.15.40:0/322590067 conn(0x7f718c0b1850 msgr2=0x7f718c0af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 -- 172.21.15.40:0/322590067 shutdown_connections 2024-03-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.595+0000 7f7192800640 1 -- 172.21.15.40:0/322590067 wait complete. 2024-03-08T06:04:44.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:44.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.842+0000 7f3acbe00640 1 Processor -- start 2024-03-08T06:04:44.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.842+0000 7f3acbe00640 1 -- start start 2024-03-08T06:04:44.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.844+0000 7f3acbe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3ac4071bc0 0x7f3ac406e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.844+0000 7f3acbe00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ac4071fa0 0x7f3ac4161c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.844+0000 7f3acbe00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac40b3120 con 0x7f3ac41621d0 2024-03-08T06:04:44.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.844+0000 7f3acbe00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40b3400 con 0x7f3ac4071bc0 2024-03-08T06:04:44.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.844+0000 7f3acbe00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40b36e0 con 0x7f3ac4071fa0 2024-03-08T06:04:44.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.844+0000 7f3ac9400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3ac4071bc0 0x7f3ac406e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:44.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.845+0000 7f3ac9e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3ac41621d0 0x7f3ac4165e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42680/0 (socket says 172.21.15.40:42680) 2024-03-08T06:04:44.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.845+0000 7f3ac9e00640 1 -- 172.21.15.40:0/4107362537 learned_addr learned my addr 172.21.15.40:0/4107362537 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:44.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.845+0000 7f3ac8a00640 1 --2- 172.21.15.40:0/4107362537 >> v2:172.21.15.53:3300/0 conn(0x7f3ac4071fa0 0x7f3ac4161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:44.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.845+0000 7f3ac3e00640 1 -- 172.21.15.40:0/4107362537 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936596449 0 0) 0x7f3ac40b3120 con 0x7f3ac41621d0 2024-03-08T06:04:44.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.845+0000 7f3ac3e00640 1 -- 172.21.15.40:0/4107362537 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aac003520 con 0x7f3ac41621d0 2024-03-08T06:04:44.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.845+0000 7f3ac9400640 1 -- 172.21.15.40:0/4107362537 >> v2:172.21.15.53:3300/0 conn(0x7f3ac4071fa0 msgr2=0x7f3ac4161c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:44.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.845+0000 7f3ac9400640 1 --2- 172.21.15.40:0/4107362537 >> v2:172.21.15.53:3300/0 conn(0x7f3ac4071fa0 0x7f3ac4161c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.845+0000 7f3ac9400640 1 -- 172.21.15.40:0/4107362537 >> v1:172.21.15.40:6789/0 conn(0x7f3ac41621d0 legacy=0x7f3ac4165e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.845+0000 7f3ac9400640 1 -- 172.21.15.40:0/4107362537 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac40b39c0 con 0x7f3ac4071bc0 2024-03-08T06:04:44.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.846+0000 7f3ac9400640 1 --2- 172.21.15.40:0/4107362537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ac4071bc0 0x7f3ac406e190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4072770 tx=0x7f3ab802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238b46360b30955e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:44.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.847+0000 7f3ac3e00640 1 -- 172.21.15.40:0/4107362537 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8039070 con 0x7f3ac4071bc0 2024-03-08T06:04:44.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.847+0000 7f3ac3e00640 1 -- 172.21.15.40:0/4107362537 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab802cc70 con 0x7f3ac4071bc0 2024-03-08T06:04:44.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.847+0000 7f3ac3e00640 1 -- 172.21.15.40:0/4107362537 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8034640 con 0x7f3ac4071bc0 2024-03-08T06:04:44.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.847+0000 7f3acbe00640 1 -- 172.21.15.40:0/4107362537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ac4071bc0 msgr2=0x7f3ac406e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.847+0000 7f3acbe00640 1 --2- 172.21.15.40:0/4107362537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ac4071bc0 0x7f3ac406e190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4072770 tx=0x7f3ab802bec0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.848+0000 7f3acbe00640 1 -- 172.21.15.40:0/4107362537 shutdown_connections 2024-03-08T06:04:44.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.848+0000 7f3acbe00640 1 --2- 172.21.15.40:0/4107362537 >> v2:172.21.15.53:3300/0 conn(0x7f3ac4071fa0 0x7f3ac4161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.848+0000 7f3acbe00640 1 --2- 172.21.15.40:0/4107362537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ac4071bc0 0x7f3ac406e190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.848+0000 7f3acbe00640 1 -- 172.21.15.40:0/4107362537 >> 172.21.15.40:0/4107362537 conn(0x7f3ac406ae30 msgr2=0x7f3ac406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:44.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.848+0000 7f3acbe00640 1 -- 172.21.15.40:0/4107362537 shutdown_connections 2024-03-08T06:04:44.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.848+0000 7f3acbe00640 1 -- 172.21.15.40:0/4107362537 wait complete. 2024-03-08T06:04:44.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.849+0000 7f3acbe00640 1 Processor -- start 2024-03-08T06:04:44.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.850+0000 7f3acbe00640 1 -- start start 2024-03-08T06:04:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.850+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac4071bc0 0x7f3ac415e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ac4071fa0 0x7f3ac413fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ac41621d0 0x7f3ac40b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3acbe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac4162620 con 0x7f3ac41621d0 2024-03-08T06:04:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3acbe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac406e6d0 con 0x7f3ac4071bc0 2024-03-08T06:04:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3acbe00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40b2cb0 con 0x7f3ac4071fa0 2024-03-08T06:04:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3ac9e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ac41621d0 0x7f3ac40b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3ac9e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ac41621d0 0x7f3ac40b0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45456/0 (socket says 172.21.15.40:45456) 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3ac9e00640 1 -- 172.21.15.40:0/3929971399 learned_addr learned my addr 172.21.15.40:0/3929971399 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3ac9400640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac4071bc0 0x7f3ac415e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3ac8a00640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ac4071fa0 0x7f3ac413fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3ac9e00640 1 -- 172.21.15.40:0/3929971399 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ac4071fa0 msgr2=0x7f3ac413fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3ac9e00640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ac4071fa0 0x7f3ac413fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3ac9e00640 1 -- 172.21.15.40:0/3929971399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac4071bc0 msgr2=0x7f3ac415e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3ac9e00640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac4071bc0 0x7f3ac415e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3ac9e00640 1 -- 172.21.15.40:0/3929971399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac415eb80 con 0x7f3ac41621d0 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.851+0000 7f3ac8a00640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ac4071fa0 0x7f3ac413fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.852+0000 7f3ac9e00640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ac41621d0 0x7f3ac40b0a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c390 tx=0x7f3abc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.852+0000 7f3ac1600640 1 -- 172.21.15.40:0/3929971399 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc017070 con 0x7f3ac41621d0 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.852+0000 7f3ac1600640 1 -- 172.21.15.40:0/3929971399 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc005ce0 con 0x7f3ac41621d0 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.852+0000 7f3acbe00640 1 -- 172.21.15.40:0/3929971399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac415ee60 con 0x7f3ac41621d0 2024-03-08T06:04:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.852+0000 7f3ac1600640 1 -- 172.21.15.40:0/3929971399 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc007310 con 0x7f3ac41621d0 2024-03-08T06:04:44.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.852+0000 7f3acbe00640 1 -- 172.21.15.40:0/3929971399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac415f420 con 0x7f3ac41621d0 2024-03-08T06:04:44.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.853+0000 7f3acbe00640 1 -- 172.21.15.40:0/3929971399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac4067cb0 con 0x7f3ac41621d0 2024-03-08T06:04:44.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.854+0000 7f3ac1600640 1 -- 172.21.15.40:0/3929971399 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3abc01a070 con 0x7f3ac41621d0 2024-03-08T06:04:44.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.855+0000 7f3ac1600640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3aa406b4d0 0x7f3aa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:44.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.855+0000 7f3ac1600640 1 -- 172.21.15.40:0/3929971399 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3abc089680 con 0x7f3ac41621d0 2024-03-08T06:04:44.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.856+0000 7f3ac9400640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3aa406b4d0 0x7f3aa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:44.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.856+0000 7f3ac9400640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3aa406b4d0 0x7f3aa406d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8036000 tx=0x7f3ab8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:44.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:44.860+0000 7f3ac1600640 1 -- 172.21.15.40:0/3929971399 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3abc058460 con 0x7f3ac41621d0 2024-03-08T06:04:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.056+0000 7f3acbe00640 1 -- 172.21.15.40:0/3929971399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ac406d780 con 0x7f3ac41621d0 2024-03-08T06:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.057+0000 7f3ac1600640 1 -- 172.21.15.40:0/3929971399 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3abc057d60 con 0x7f3ac41621d0 2024-03-08T06:04:45.059 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 -- 172.21.15.40:0/3929971399 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3aa406b4d0 msgr2=0x7f3aa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3aa406b4d0 0x7f3aa406d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8036000 tx=0x7f3ab8030000 comp rx=0 tx=0).stop 2024-03-08T06:04:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 -- 172.21.15.40:0/3929971399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ac41621d0 msgr2=0x7f3ac40b0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ac41621d0 0x7f3ac40b0a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c390 tx=0x7f3abc00c850 comp rx=0 tx=0).stop 2024-03-08T06:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 -- 172.21.15.40:0/3929971399 shutdown_connections 2024-03-08T06:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3aa406b4d0 0x7f3aa406d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ac41621d0 0x7f3ac40b0a00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ac4071fa0 0x7f3ac413fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 --2- 172.21.15.40:0/3929971399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac4071bc0 0x7f3ac415e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 -- 172.21.15.40:0/3929971399 >> 172.21.15.40:0/3929971399 conn(0x7f3ac406ae30 msgr2=0x7f3ac40b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 -- 172.21.15.40:0/3929971399 shutdown_connections 2024-03-08T06:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.060+0000 7f3acbe00640 1 -- 172.21.15.40:0/3929971399 wait complete. 2024-03-08T06:04:45.082 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:45.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.301+0000 7ff03fe00640 1 Processor -- start 2024-03-08T06:04:45.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.302+0000 7ff03fe00640 1 -- start start 2024-03-08T06:04:45.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.303+0000 7ff03fe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff040144240 0x7ff0400cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.303+0000 7ff03fe00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0400cd3e0 con 0x7ff04006ea40 2024-03-08T06:04:45.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.303+0000 7ff03fe00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0400b37e0 con 0x7ff0400b42c0 2024-03-08T06:04:45.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.303+0000 7ff03fe00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0400b3e30 con 0x7ff040144240 2024-03-08T06:04:45.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.304+0000 7ff03ea00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff040144240 0x7ff0400cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:45.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.304+0000 7ff03f400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff04006ea40 0x7ff04015e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42694/0 (socket says 172.21.15.40:42694) 2024-03-08T06:04:45.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.304+0000 7ff03ea00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff040144240 0x7ff0400cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45464/0 (socket says 172.21.15.40:45464) 2024-03-08T06:04:45.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.304+0000 7ff03f400640 1 -- 172.21.15.40:0/2488107837 learned_addr learned my addr 172.21.15.40:0/2488107837 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:45.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.304+0000 7ff03ea00640 1 -- 172.21.15.40:0/2488107837 >> v1:172.21.15.53:6789/0 conn(0x7ff0400b42c0 legacy=0x7ff04006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:45.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.304+0000 7ff03e000640 1 -- 172.21.15.40:0/2488107837 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610824692 0 0) 0x7ff0400cd3e0 con 0x7ff04006ea40 2024-03-08T06:04:45.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.304+0000 7ff03ea00640 1 -- 172.21.15.40:0/2488107837 >> v1:172.21.15.40:6789/0 conn(0x7ff04006ea40 legacy=0x7ff04015e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.304+0000 7ff03ea00640 1 -- 172.21.15.40:0/2488107837 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0400b3ac0 con 0x7ff040144240 2024-03-08T06:04:45.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.305+0000 7ff03ea00640 1 --2- 172.21.15.40:0/2488107837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff040144240 0x7ff0400cc9b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff0300097a0 tx=0x7ff03002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cde73432d4b4877 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:45.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.306+0000 7ff03e000640 1 -- 172.21.15.40:0/2488107837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0300315a0 con 0x7ff040144240 2024-03-08T06:04:45.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.306+0000 7ff03e000640 1 -- 172.21.15.40:0/2488107837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff030037070 con 0x7ff040144240 2024-03-08T06:04:45.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.306+0000 7ff03e000640 1 -- 172.21.15.40:0/2488107837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030031960 con 0x7ff040144240 2024-03-08T06:04:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.306+0000 7ff03fe00640 1 -- 172.21.15.40:0/2488107837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff040144240 msgr2=0x7ff0400cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.306+0000 7ff03fe00640 1 --2- 172.21.15.40:0/2488107837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff040144240 0x7ff0400cc9b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff0300097a0 tx=0x7ff03002b840 comp rx=0 tx=0).stop 2024-03-08T06:04:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.306+0000 7ff03fe00640 1 -- 172.21.15.40:0/2488107837 shutdown_connections 2024-03-08T06:04:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.306+0000 7ff03fe00640 1 --2- 172.21.15.40:0/2488107837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff040144240 0x7ff0400cc9b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.306+0000 7ff03fe00640 1 -- 172.21.15.40:0/2488107837 >> 172.21.15.40:0/2488107837 conn(0x7ff0400b1850 msgr2=0x7ff0400b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.307+0000 7ff03fe00640 1 -- 172.21.15.40:0/2488107837 shutdown_connections 2024-03-08T06:04:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.307+0000 7ff03fe00640 1 -- 172.21.15.40:0/2488107837 wait complete. 2024-03-08T06:04:45.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.308+0000 7ff03fe00640 1 Processor -- start 2024-03-08T06:04:45.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.309+0000 7ff03fe00640 1 -- start start 2024-03-08T06:04:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.309+0000 7ff03fe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff04006ea40 0x7ff04013aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.309+0000 7ff03fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff040144240 0x7ff0400caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.309+0000 7ff03fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff04013c7c0 0x7ff0401f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.309+0000 7ff03fe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400b4b30 con 0x7ff04013c7c0 2024-03-08T06:04:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.309+0000 7ff03fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400cd6c0 con 0x7ff040144240 2024-03-08T06:04:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.309+0000 7ff03fe00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400b4cb0 con 0x7ff04006ea40 2024-03-08T06:04:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.310+0000 7ff03f400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff04013c7c0 0x7ff0401f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.310+0000 7ff03f400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff04013c7c0 0x7ff0401f7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45466/0 (socket says 172.21.15.40:45466) 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.310+0000 7ff03f400640 1 -- 172.21.15.40:0/3313857098 learned_addr learned my addr 172.21.15.40:0/3313857098 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.310+0000 7ff03ea00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff04006ea40 0x7ff04013aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.310+0000 7ff037e00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff040144240 0x7ff0400caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.310+0000 7ff03f400640 1 -- 172.21.15.40:0/3313857098 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff04006ea40 msgr2=0x7ff04013aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.310+0000 7ff03f400640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff04006ea40 0x7ff04013aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.310+0000 7ff03f400640 1 -- 172.21.15.40:0/3313857098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff040144240 msgr2=0x7ff0400caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.310+0000 7ff03f400640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff040144240 0x7ff0400caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.310+0000 7ff03f400640 1 -- 172.21.15.40:0/3313857098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0401f7b00 con 0x7ff04013c7c0 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.310+0000 7ff03ea00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff04006ea40 0x7ff04013aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.311+0000 7ff03f400640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff04013c7c0 0x7ff0401f7120 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff03800c780 tx=0x7ff03800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.311+0000 7ff036a00640 1 -- 172.21.15.40:0/3313857098 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038018070 con 0x7ff04013c7c0 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.311+0000 7ff036a00640 1 -- 172.21.15.40:0/3313857098 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03800a640 con 0x7ff04013c7c0 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.311+0000 7ff036a00640 1 -- 172.21.15.40:0/3313857098 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03800aab0 con 0x7ff04013c7c0 2024-03-08T06:04:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.311+0000 7ff03fe00640 1 -- 172.21.15.40:0/3313857098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0401f7de0 con 0x7ff04013c7c0 2024-03-08T06:04:45.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.311+0000 7ff03fe00640 1 -- 172.21.15.40:0/3313857098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04013a540 con 0x7ff04013c7c0 2024-03-08T06:04:45.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.312+0000 7ff03fe00640 1 -- 172.21.15.40:0/3313857098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04006ee20 con 0x7ff04013c7c0 2024-03-08T06:04:45.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.314+0000 7ff036a00640 1 -- 172.21.15.40:0/3313857098 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff03800ac50 con 0x7ff04013c7c0 2024-03-08T06:04:45.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.315+0000 7ff036a00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff01c06b4d0 0x7ff01c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.315+0000 7ff036a00640 1 -- 172.21.15.40:0/3313857098 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff038089900 con 0x7ff04013c7c0 2024-03-08T06:04:45.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.316+0000 7ff03ea00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff01c06b4d0 0x7ff01c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:45.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.316+0000 7ff03ea00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff01c06b4d0 0x7ff01c06d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff030036000 tx=0x7ff030008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:45.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.320+0000 7ff036a00640 1 -- 172.21.15.40:0/3313857098 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff04006ee20 con 0x7ff04013c7c0 2024-03-08T06:04:45.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.519+0000 7ff03fe00640 1 -- 172.21.15.40:0/3313857098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff040075090 con 0x7ff04013c7c0 2024-03-08T06:04:45.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.519+0000 7ff036a00640 1 -- 172.21.15.40:0/3313857098 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff038089030 con 0x7ff04013c7c0 2024-03-08T06:04:45.521 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:45.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 -- 172.21.15.40:0/3313857098 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff01c06b4d0 msgr2=0x7ff01c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff01c06b4d0 0x7ff01c06d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff030036000 tx=0x7ff030008030 comp rx=0 tx=0).stop 2024-03-08T06:04:45.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 -- 172.21.15.40:0/3313857098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff04013c7c0 msgr2=0x7ff0401f7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff04013c7c0 0x7ff0401f7120 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff03800c780 tx=0x7ff03800cc40 comp rx=0 tx=0).stop 2024-03-08T06:04:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 -- 172.21.15.40:0/3313857098 shutdown_connections 2024-03-08T06:04:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff01c06b4d0 0x7ff01c06d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff04013c7c0 0x7ff0401f7120 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff040144240 0x7ff0400caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 --2- 172.21.15.40:0/3313857098 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff04006ea40 0x7ff04013aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 -- 172.21.15.40:0/3313857098 >> 172.21.15.40:0/3313857098 conn(0x7ff0400b1850 msgr2=0x7ff0400ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:45.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 -- 172.21.15.40:0/3313857098 shutdown_connections 2024-03-08T06:04:45.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.522+0000 7ff03fe00640 1 -- 172.21.15.40:0/3313857098 wait complete. 2024-03-08T06:04:45.542 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:04:45.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.765+0000 7f9839e00640 1 Processor -- start 2024-03-08T06:04:45.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.765+0000 7f9839e00640 1 -- start start 2024-03-08T06:04:45.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.767+0000 7f9839e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f98341451b0 0x7f983415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.767+0000 7f9839e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9834071b10 0x7f983415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.767+0000 7f9839e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983415cf70 con 0x7f983406f050 2024-03-08T06:04:45.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.767+0000 7f9839e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983406d600 con 0x7f98341451b0 2024-03-08T06:04:45.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.767+0000 7f9839e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983406d8e0 con 0x7f9834071b10 2024-03-08T06:04:45.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.767+0000 7f9833e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f983406f050 0x7f9834071400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42696/0 (socket says 172.21.15.40:42696) 2024-03-08T06:04:45.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.767+0000 7f9833e00640 1 -- 172.21.15.40:0/799588581 learned_addr learned my addr 172.21.15.40:0/799588581 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:45.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9838a00640 1 --2- 172.21.15.40:0/799588581 >> v2:172.21.15.53:3300/0 conn(0x7f98341451b0 0x7f983415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:45.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9839400640 1 --2- 172.21.15.40:0/799588581 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9834071b10 0x7f983415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960444945 0 0) 0x7f983415cf70 con 0x7f983406f050 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f981c003520 con 0x7f983406f050 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3342172367 0 0) 0x7f981c003520 con 0x7f983406f050 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9834071b10 msgr2=0x7f983415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 --2- 172.21.15.40:0/799588581 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9834071b10 0x7f983415d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 >> v2:172.21.15.53:3300/0 conn(0x7f98341451b0 msgr2=0x7f983415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 --2- 172.21.15.40:0/799588581 >> v2:172.21.15.53:3300/0 conn(0x7f98341451b0 0x7f983415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983406dbc0 con 0x7f983406f050 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f9820001670 con 0x7f983406f050 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 >> v1:172.21.15.40:6789/0 conn(0x7f983406f050 legacy=0x7f9834071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 --2- 172.21.15.40:0/799588581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 0x7f981c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.768+0000 7f9833400640 1 --2- 172.21.15.40:0/799588581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f981c004800 0x7f981c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.769+0000 7f9833400640 1 --2- 172.21.15.40:0/799588581 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9834071b10 0x7f981c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.769+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983406d600 con 0x7f981c003a90 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.769+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983406d8e0 con 0x7f981c004800 2024-03-08T06:04:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.769+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c00ba70 con 0x7f9834071b10 2024-03-08T06:04:45.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.769+0000 7f9838a00640 1 --2- 172.21.15.40:0/799588581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 0x7f981c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:45.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.769+0000 7f9838a00640 1 -- 172.21.15.40:0/799588581 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9834071b10 msgr2=0x7f981c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:45.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.769+0000 7f9838a00640 1 --2- 172.21.15.40:0/799588581 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9834071b10 0x7f981c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.769+0000 7f9838a00640 1 -- 172.21.15.40:0/799588581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f981c004800 msgr2=0x7f981c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:45.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.769+0000 7f9838a00640 1 --2- 172.21.15.40:0/799588581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f981c004800 0x7f981c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.769+0000 7f9838a00640 1 -- 172.21.15.40:0/799588581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c003520 con 0x7f981c003a90 2024-03-08T06:04:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.770+0000 7f9838a00640 1 --2- 172.21.15.40:0/799588581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 0x7f981c003eb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9828009ac0 tx=0x7f981c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bd9a2822b412df server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.771+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828039070 con 0x7f981c003a90 2024-03-08T06:04:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.771+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982802c710 con 0x7f981c003a90 2024-03-08T06:04:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.771+0000 7f9833400640 1 -- 172.21.15.40:0/799588581 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828009690 con 0x7f981c003a90 2024-03-08T06:04:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.771+0000 7f9839e00640 1 -- 172.21.15.40:0/799588581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 msgr2=0x7f981c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.771+0000 7f9839e00640 1 --2- 172.21.15.40:0/799588581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 0x7f981c003eb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9828009ac0 tx=0x7f981c005830 comp rx=0 tx=0).stop 2024-03-08T06:04:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.771+0000 7f9839400640 1 -- 172.21.15.40:0/799588581 reap_dead start 2024-03-08T06:04:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.772+0000 7f9839e00640 1 -- 172.21.15.40:0/799588581 shutdown_connections 2024-03-08T06:04:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.772+0000 7f9839e00640 1 -- 172.21.15.40:0/799588581 >> 172.21.15.40:0/799588581 conn(0x7f98340b1590 msgr2=0x7f98340b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.772+0000 7f9839e00640 1 -- 172.21.15.40:0/799588581 shutdown_connections 2024-03-08T06:04:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.772+0000 7f9839e00640 1 -- 172.21.15.40:0/799588581 wait complete. 2024-03-08T06:04:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.773+0000 7f9839e00640 1 Processor -- start 2024-03-08T06:04:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.774+0000 7f9839e00640 1 -- start start 2024-03-08T06:04:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.774+0000 7f9839e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 0x7f98340c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.774+0000 7f9839e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98341f4360 0x7f983415fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.774+0000 7f9839e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98340c6c40 0x7f98340c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9839e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9834145710 con 0x7f981c003a90 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9839e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9834145590 con 0x7f98340c6c40 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9839e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983415d1a0 con 0x7f98341f4360 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9838a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 0x7f98340c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9838a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 0x7f98340c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45482/0 (socket says 172.21.15.40:45482) 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9838a00640 1 -- 172.21.15.40:0/173381750 learned_addr learned my addr 172.21.15.40:0/173381750 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9838a00640 1 -- 172.21.15.40:0/173381750 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98341f4360 msgr2=0x7f983415fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9838a00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98341f4360 0x7f983415fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9839400640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98340c6c40 0x7f98340c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9838a00640 1 -- 172.21.15.40:0/173381750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98340c6c40 msgr2=0x7f98340c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9838a00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98340c6c40 0x7f98340c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.775+0000 7f9838a00640 1 -- 172.21.15.40:0/173381750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340c4b70 con 0x7f981c003a90 2024-03-08T06:04:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.776+0000 7f9838a00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 0x7f98340c5c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9828037040 tx=0x7f9828004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.776+0000 7f9830c00640 1 -- 172.21.15.40:0/173381750 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828042020 con 0x7f981c003a90 2024-03-08T06:04:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.776+0000 7f9830c00640 1 -- 172.21.15.40:0/173381750 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982803a040 con 0x7f981c003a90 2024-03-08T06:04:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.776+0000 7f9830c00640 1 -- 172.21.15.40:0/173381750 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828039070 con 0x7f981c003a90 2024-03-08T06:04:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.776+0000 7f9839e00640 1 -- 172.21.15.40:0/173381750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98340c4e50 con 0x7f981c003a90 2024-03-08T06:04:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.776+0000 7f9839e00640 1 -- 172.21.15.40:0/173381750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98340c5410 con 0x7f981c003a90 2024-03-08T06:04:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.777+0000 7f9839e00640 1 -- 172.21.15.40:0/173381750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9834075090 con 0x7f981c003a90 2024-03-08T06:04:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.778+0000 7f9830c00640 1 -- 172.21.15.40:0/173381750 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9828045020 con 0x7f981c003a90 2024-03-08T06:04:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.779+0000 7f9830c00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f981406b4d0 0x7f981406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.779+0000 7f9830c00640 1 -- 172.21.15.40:0/173381750 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98280ab670 con 0x7f981c003a90 2024-03-08T06:04:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.779+0000 7f9833e00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f981406b4d0 0x7f981406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.780+0000 7f9833e00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f981406b4d0 0x7f981406d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f98341f7f90 tx=0x7f98200014d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.783+0000 7f9830c00640 1 -- 172.21.15.40:0/173381750 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f982804c070 con 0x7f981c003a90 2024-03-08T06:04:45.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.992+0000 7f9839e00640 1 -- 172.21.15.40:0/173381750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9834143d30 con 0x7f981c003a90 2024-03-08T06:04:45.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.993+0000 7f9830c00640 1 -- 172.21.15.40:0/173381750 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f982807a450 con 0x7f981c003a90 2024-03-08T06:04:45.994 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.995+0000 7f9839e00640 1 -- 172.21.15.40:0/173381750 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f981406b4d0 msgr2=0x7f981406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.995+0000 7f9839e00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f981406b4d0 0x7f981406d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f98341f7f90 tx=0x7f98200014d0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.995+0000 7f9839e00640 1 -- 172.21.15.40:0/173381750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 msgr2=0x7f98340c5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.995+0000 7f9839e00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 0x7f98340c5c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9828037040 tx=0x7f9828004730 comp rx=0 tx=0).stop 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.995+0000 7f9839e00640 1 -- 172.21.15.40:0/173381750 shutdown_connections 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.995+0000 7f9839e00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f981406b4d0 0x7f981406d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.995+0000 7f9839e00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98340c6c40 0x7f98340c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.995+0000 7f9839e00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98341f4360 0x7f983415fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.995+0000 7f9839e00640 1 --2- 172.21.15.40:0/173381750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f981c003a90 0x7f98340c5c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.995+0000 7f9839e00640 1 -- 172.21.15.40:0/173381750 >> 172.21.15.40:0/173381750 conn(0x7f98340b1590 msgr2=0x7f98340b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.995+0000 7f9839e00640 1 -- 172.21.15.40:0/173381750 shutdown_connections 2024-03-08T06:04:45.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:45.996+0000 7f9839e00640 1 -- 172.21.15.40:0/173381750 wait complete. 2024-03-08T06:04:46.016 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-08T06:04:46.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:46.016 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-08T06:04:46.120 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:04:46.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:46.121 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-08T06:04:46.183 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:04:46.183 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:46.244 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:04:46.244 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:46.303 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:04:46.304 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:46.359 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:46.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.587+0000 7f5e4a400640 1 Processor -- start 2024-03-08T06:04:46.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.588+0000 7f5e4a400640 1 -- start start 2024-03-08T06:04:46.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.589+0000 7f5e4a400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e44071ab0 0x7f5e4415cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:46.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.589+0000 7f5e4a400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e4406d600 0x7f5e44144d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.589+0000 7f5e4a400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5e44071e90 0x7f5e441452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.589+0000 7f5e4a400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4406ef40 con 0x7f5e4406d600 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.589+0000 7f5e4a400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e441457e0 con 0x7f5e44071e90 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.589+0000 7f5e4a400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4415d0c0 con 0x7f5e44071ab0 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.590+0000 7f5e3be00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e4406d600 0x7f5e44144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.590+0000 7f5e3be00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e4406d600 0x7f5e44144d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45484/0 (socket says 172.21.15.40:45484) 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.590+0000 7f5e3be00640 1 -- 172.21.15.40:0/232023008 learned_addr learned my addr 172.21.15.40:0/232023008 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.590+0000 7f5e3be00640 1 -- 172.21.15.40:0/232023008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e44071ab0 msgr2=0x7f5e4415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.590+0000 7f5e3be00640 1 --2- 172.21.15.40:0/232023008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e44071ab0 0x7f5e4415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.590+0000 7f5e3be00640 1 -- 172.21.15.40:0/232023008 >> v2:172.21.15.53:3300/0 conn(0x7f5e44071e90 msgr2=0x7f5e441452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.590+0000 7f5e3be00640 1 --2- 172.21.15.40:0/232023008 >> v2:172.21.15.53:3300/0 conn(0x7f5e44071e90 0x7f5e441452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.590+0000 7f5e3be00640 1 -- 172.21.15.40:0/232023008 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4406da80 con 0x7f5e4406d600 2024-03-08T06:04:46.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.591+0000 7f5e3be00640 1 --2- 172.21.15.40:0/232023008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e4406d600 0x7f5e44144d60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5e4415d270 tx=0x7f5e2c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516498e90154c870 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:46.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.592+0000 7f5e42a00640 1 -- 172.21.15.40:0/232023008 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c035020 con 0x7f5e4406d600 2024-03-08T06:04:46.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.592+0000 7f5e42a00640 1 -- 172.21.15.40:0/232023008 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c031640 con 0x7f5e4406d600 2024-03-08T06:04:46.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.592+0000 7f5e42a00640 1 -- 172.21.15.40:0/232023008 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c02d040 con 0x7f5e4406d600 2024-03-08T06:04:46.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.593+0000 7f5e4a400640 1 -- 172.21.15.40:0/232023008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e4406d600 msgr2=0x7f5e44144d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.593+0000 7f5e4a400640 1 --2- 172.21.15.40:0/232023008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e4406d600 0x7f5e44144d60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5e4415d270 tx=0x7f5e2c02b8f0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.593+0000 7f5e4a400640 1 -- 172.21.15.40:0/232023008 shutdown_connections 2024-03-08T06:04:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.593+0000 7f5e4a400640 1 --2- 172.21.15.40:0/232023008 >> v2:172.21.15.53:3300/0 conn(0x7f5e44071e90 0x7f5e441452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.593+0000 7f5e4a400640 1 --2- 172.21.15.40:0/232023008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e4406d600 0x7f5e44144d60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.593+0000 7f5e4a400640 1 --2- 172.21.15.40:0/232023008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e44071ab0 0x7f5e4415cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.593+0000 7f5e4a400640 1 -- 172.21.15.40:0/232023008 >> 172.21.15.40:0/232023008 conn(0x7f5e4406ae30 msgr2=0x7f5e4406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.593+0000 7f5e4a400640 1 -- 172.21.15.40:0/232023008 shutdown_connections 2024-03-08T06:04:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.593+0000 7f5e4a400640 1 -- 172.21.15.40:0/232023008 wait complete. 2024-03-08T06:04:46.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.595+0000 7f5e4a400640 1 Processor -- start 2024-03-08T06:04:46.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.595+0000 7f5e4a400640 1 -- start start 2024-03-08T06:04:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.596+0000 7f5e4a400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e4406d600 0x7f5e4414dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.596+0000 7f5e4a400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e44071ab0 0x7f5e4413f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.596+0000 7f5e4a400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e44071e90 0x7f5e4415d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.596+0000 7f5e4a400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4406e320 con 0x7f5e44071ab0 2024-03-08T06:04:46.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.596+0000 7f5e4a400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4406e040 con 0x7f5e44071e90 2024-03-08T06:04:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.596+0000 7f5e4a400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4406e600 con 0x7f5e4406d600 2024-03-08T06:04:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.596+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e44071ab0 0x7f5e4413f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.596+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e44071ab0 0x7f5e4413f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45494/0 (socket says 172.21.15.40:45494) 2024-03-08T06:04:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.596+0000 7f5e3be00640 1 -- 172.21.15.40:0/1414378698 learned_addr learned my addr 172.21.15.40:0/1414378698 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:46.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.596+0000 7f5e43e00640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e44071e90 0x7f5e4415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:46.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e43400640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e4406d600 0x7f5e4414dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:46.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e3be00640 1 -- 172.21.15.40:0/1414378698 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e4406d600 msgr2=0x7f5e4414dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:46.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e3be00640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e4406d600 0x7f5e4414dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e3be00640 1 -- 172.21.15.40:0/1414378698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e44071e90 msgr2=0x7f5e4415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:46.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e3be00640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e44071e90 0x7f5e4415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e3be00640 1 -- 172.21.15.40:0/1414378698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e44140300 con 0x7f5e44071ab0 2024-03-08T06:04:46.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e43400640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e4406d600 0x7f5e4414dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:46.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e3be00640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e44071ab0 0x7f5e4413f920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c000c00 tx=0x7f5e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:46.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e3b400640 1 -- 172.21.15.40:0/1414378698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c03ac60 con 0x7f5e44071ab0 2024-03-08T06:04:46.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e3b400640 1 -- 172.21.15.40:0/1414378698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c03ae20 con 0x7f5e44071ab0 2024-03-08T06:04:46.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e3b400640 1 -- 172.21.15.40:0/1414378698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c044630 con 0x7f5e44071ab0 2024-03-08T06:04:46.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.597+0000 7f5e4a400640 1 -- 172.21.15.40:0/1414378698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e44140020 con 0x7f5e44071ab0 2024-03-08T06:04:46.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.598+0000 7f5e4a400640 1 -- 172.21.15.40:0/1414378698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4414b690 con 0x7f5e44071ab0 2024-03-08T06:04:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.599+0000 7f5e4a400640 1 -- 172.21.15.40:0/1414378698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e44067cb0 con 0x7f5e44071ab0 2024-03-08T06:04:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.600+0000 7f5e3b400640 1 -- 172.21.15.40:0/1414378698 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e2c04b060 con 0x7f5e44071ab0 2024-03-08T06:04:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.601+0000 7f5e3b400640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2006b4d0 0x7f5e2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.601+0000 7f5e3b400640 1 -- 172.21.15.40:0/1414378698 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e2c0ac0b0 con 0x7f5e44071ab0 2024-03-08T06:04:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.601+0000 7f5e43400640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2006b4d0 0x7f5e2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.602+0000 7f5e43400640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2006b4d0 0x7f5e2006d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e34006fd0 tx=0x7f5e34008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:46.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.605+0000 7f5e3b400640 1 -- 172.21.15.40:0/1414378698 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e2c03f070 con 0x7f5e44071ab0 2024-03-08T06:04:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.805+0000 7f5e4a400640 1 -- 172.21.15.40:0/1414378698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e4415e5a0 con 0x7f5e44071ab0 2024-03-08T06:04:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.807+0000 7f5e3b400640 1 -- 172.21.15.40:0/1414378698 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f5e2c07ae00 con 0x7f5e44071ab0 2024-03-08T06:04:46.809 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:46.809 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:03.505178+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.810+0000 7f5e4a400640 1 -- 172.21.15.40:0/1414378698 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2006b4d0 msgr2=0x7f5e2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.810+0000 7f5e4a400640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2006b4d0 0x7f5e2006d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e34006fd0 tx=0x7f5e34008040 comp rx=0 tx=0).stop 2024-03-08T06:04:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.810+0000 7f5e4a400640 1 -- 172.21.15.40:0/1414378698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e44071ab0 msgr2=0x7f5e4413f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.810+0000 7f5e4a400640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e44071ab0 0x7f5e4413f920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c000c00 tx=0x7f5e2c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.811+0000 7f5e4a400640 1 -- 172.21.15.40:0/1414378698 shutdown_connections 2024-03-08T06:04:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.811+0000 7f5e4a400640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2006b4d0 0x7f5e2006d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.811+0000 7f5e4a400640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e44071e90 0x7f5e4415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.811+0000 7f5e4a400640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e44071ab0 0x7f5e4413f920 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.811+0000 7f5e4a400640 1 --2- 172.21.15.40:0/1414378698 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e4406d600 0x7f5e4414dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.811+0000 7f5e4a400640 1 -- 172.21.15.40:0/1414378698 >> 172.21.15.40:0/1414378698 conn(0x7f5e4406ae30 msgr2=0x7f5e44144b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.811+0000 7f5e4a400640 1 -- 172.21.15.40:0/1414378698 shutdown_connections 2024-03-08T06:04:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:46.811+0000 7f5e4a400640 1 -- 172.21.15.40:0/1414378698 wait complete. 2024-03-08T06:04:46.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:46.834 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:47.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.060+0000 7faf07000640 1 Processor -- start 2024-03-08T06:04:47.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.061+0000 7faf07000640 1 -- start start 2024-03-08T06:04:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.062+0000 7faf07000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 0x7faf000b9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.062+0000 7faf07000640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faf000d3040 0x7faf000ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.062+0000 7faf07000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf000d1fa0 con 0x7faf000d34c0 2024-03-08T06:04:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.062+0000 7faf07000640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf000d2280 con 0x7faf000d3040 2024-03-08T06:04:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.062+0000 7faf07000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf000d2560 con 0x7faf0015de00 2024-03-08T06:04:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.063+0000 7faf05c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 0x7faf000b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.063+0000 7faf05c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 0x7faf000b9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:37730/0 (socket says 172.21.15.40:37730) 2024-03-08T06:04:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.063+0000 7faf05c00640 1 -- 172.21.15.40:0/1570167996 learned_addr learned my addr 172.21.15.40:0/1570167996 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.063+0000 7faf05200640 1 --2- 172.21.15.40:0/1570167996 >> v2:172.21.15.53:3300/0 conn(0x7faf000d3040 0x7faf000ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.064+0000 7faf05c00640 1 -- 172.21.15.40:0/1570167996 >> v2:172.21.15.53:3300/0 conn(0x7faf000d3040 msgr2=0x7faf000ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.064+0000 7faf05c00640 1 --2- 172.21.15.40:0/1570167996 >> v2:172.21.15.53:3300/0 conn(0x7faf000d3040 0x7faf000ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.064+0000 7faf05c00640 1 -- 172.21.15.40:0/1570167996 >> v1:172.21.15.53:6789/0 conn(0x7faf000d34c0 legacy=0x7faf000b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.064+0000 7faf05c00640 1 -- 172.21.15.40:0/1570167996 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf000d2840 con 0x7faf0015de00 2024-03-08T06:04:47.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.065+0000 7faf05c00640 1 --2- 172.21.15.40:0/1570167996 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 0x7faf000b9410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faef0009e50 tx=0x7faef002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2624fa1c72ad8c2a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:47.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.066+0000 7faeffe00640 1 -- 172.21.15.40:0/1570167996 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef002ca80 con 0x7faf0015de00 2024-03-08T06:04:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.066+0000 7faeffe00640 1 -- 172.21.15.40:0/1570167996 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef002cc20 con 0x7faf0015de00 2024-03-08T06:04:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.066+0000 7faeffe00640 1 -- 172.21.15.40:0/1570167996 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef00343f0 con 0x7faf0015de00 2024-03-08T06:04:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.066+0000 7faf07000640 1 -- 172.21.15.40:0/1570167996 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 msgr2=0x7faf000b9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.066+0000 7faf07000640 1 --2- 172.21.15.40:0/1570167996 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 0x7faf000b9410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faef0009e50 tx=0x7faef002bee0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.066+0000 7faf07000640 1 -- 172.21.15.40:0/1570167996 shutdown_connections 2024-03-08T06:04:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.066+0000 7faf07000640 1 --2- 172.21.15.40:0/1570167996 >> v2:172.21.15.53:3300/0 conn(0x7faf000d3040 0x7faf000ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.066+0000 7faf07000640 1 --2- 172.21.15.40:0/1570167996 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 0x7faf000b9410 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.066+0000 7faf07000640 1 -- 172.21.15.40:0/1570167996 >> 172.21.15.40:0/1570167996 conn(0x7faf00066fc0 msgr2=0x7faf000b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.066+0000 7faf07000640 1 -- 172.21.15.40:0/1570167996 shutdown_connections 2024-03-08T06:04:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.066+0000 7faf07000640 1 -- 172.21.15.40:0/1570167996 wait complete. 2024-03-08T06:04:47.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.068+0000 7faf07000640 1 Processor -- start 2024-03-08T06:04:47.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.068+0000 7faf07000640 1 -- start start 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.069+0000 7faf07000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000d3040 0x7faf00152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.069+0000 7faf07000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf000d34c0 0x7faf00157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.069+0000 7faf07000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 0x7faf00159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.069+0000 7faf07000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf000d1be0 con 0x7faf000d34c0 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.069+0000 7faf07000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0015e1e0 con 0x7faf000d3040 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.069+0000 7faf07000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf000d1d60 con 0x7faf0015de00 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.069+0000 7faf05200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf000d34c0 0x7faf00157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.069+0000 7faf05200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf000d34c0 0x7faf00157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44016/0 (socket says 172.21.15.40:44016) 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.070+0000 7faf05200640 1 -- 172.21.15.40:0/1588886719 learned_addr learned my addr 172.21.15.40:0/1588886719 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.070+0000 7faf05c00640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000d3040 0x7faf00152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.070+0000 7faf06600640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 0x7faf00159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.070+0000 7faf05200640 1 -- 172.21.15.40:0/1588886719 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 msgr2=0x7faf00159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.070+0000 7faf05200640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 0x7faf00159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.070+0000 7faf05200640 1 -- 172.21.15.40:0/1588886719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000d3040 msgr2=0x7faf00152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.070+0000 7faf05200640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000d3040 0x7faf00152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.070+0000 7faf05200640 1 -- 172.21.15.40:0/1588886719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf00154180 con 0x7faf000d34c0 2024-03-08T06:04:47.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.070+0000 7faf05200640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf000d34c0 0x7faf00157470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faeec00e9b0 tx=0x7faeec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:47.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.071+0000 7faefd600640 1 -- 172.21.15.40:0/1588886719 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec017070 con 0x7faf000d34c0 2024-03-08T06:04:47.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.071+0000 7faefd600640 1 -- 172.21.15.40:0/1588886719 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec00c9a0 con 0x7faf000d34c0 2024-03-08T06:04:47.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.071+0000 7faf07000640 1 -- 172.21.15.40:0/1588886719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf00154460 con 0x7faf000d34c0 2024-03-08T06:04:47.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.071+0000 7faefd600640 1 -- 172.21.15.40:0/1588886719 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec00cd90 con 0x7faf000d34c0 2024-03-08T06:04:47.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.071+0000 7faf07000640 1 -- 172.21.15.40:0/1588886719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf00154a20 con 0x7faf000d34c0 2024-03-08T06:04:47.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.072+0000 7faf07000640 1 -- 172.21.15.40:0/1588886719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf00075090 con 0x7faf000d34c0 2024-03-08T06:04:47.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.073+0000 7faefd600640 1 -- 172.21.15.40:0/1588886719 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faeec01a070 con 0x7faf000d34c0 2024-03-08T06:04:47.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.074+0000 7faefd600640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faee006b4d0 0x7faee006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.074+0000 7faefd600640 1 -- 172.21.15.40:0/1588886719 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faeec08a2d0 con 0x7faf000d34c0 2024-03-08T06:04:47.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.074+0000 7faf05c00640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faee006b4d0 0x7faee006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.075+0000 7faf05c00640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faee006b4d0 0x7faee006d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faef0035000 tx=0x7faef0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:47.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.078+0000 7faefd600640 1 -- 172.21.15.40:0/1588886719 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faeec0590b0 con 0x7faf000d34c0 2024-03-08T06:04:47.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.281+0000 7faf07000640 1 -- 172.21.15.40:0/1588886719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf0015c2f0 con 0x7faf000d34c0 2024-03-08T06:04:47.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.282+0000 7faefd600640 1 -- 172.21.15.40:0/1588886719 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7faeec0589b0 con 0x7faf000d34c0 2024-03-08T06:04:47.283 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:47.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 -- 172.21.15.40:0/1588886719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faee006b4d0 msgr2=0x7faee006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faee006b4d0 0x7faee006d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faef0035000 tx=0x7faef0030000 comp rx=0 tx=0).stop 2024-03-08T06:04:47.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 -- 172.21.15.40:0/1588886719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf000d34c0 msgr2=0x7faf00157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf000d34c0 0x7faf00157470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faeec00e9b0 tx=0x7faeec00ee70 comp rx=0 tx=0).stop 2024-03-08T06:04:47.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 -- 172.21.15.40:0/1588886719 shutdown_connections 2024-03-08T06:04:47.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faee006b4d0 0x7faee006d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf0015de00 0x7faf00159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf000d34c0 0x7faf00157470 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 --2- 172.21.15.40:0/1588886719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000d3040 0x7faf00152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 -- 172.21.15.40:0/1588886719 >> 172.21.15.40:0/1588886719 conn(0x7faf00066fc0 msgr2=0x7faf000d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 -- 172.21.15.40:0/1588886719 shutdown_connections 2024-03-08T06:04:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.285+0000 7faf07000640 1 -- 172.21.15.40:0/1588886719 wait complete. 2024-03-08T06:04:47.305 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:47.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.531+0000 7f6e6aa00640 1 Processor -- start 2024-03-08T06:04:47.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.532+0000 7f6e6aa00640 1 -- start start 2024-03-08T06:04:47.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.533+0000 7f6e6aa00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e6415de50 0x7f6e64161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.533+0000 7f6e6aa00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6e640b97d0 0x7f6e640ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.533+0000 7f6e6aa00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e64162520 con 0x7f6e6415e230 2024-03-08T06:04:47.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.533+0000 7f6e6aa00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e64162ae0 con 0x7f6e6415de50 2024-03-08T06:04:47.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.533+0000 7f6e6aa00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640d2360 con 0x7f6e640b97d0 2024-03-08T06:04:47.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.533+0000 7f6e63e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e6415de50 0x7f6e64161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.534+0000 7f6e63e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e6415de50 0x7f6e64161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44018/0 (socket says 172.21.15.40:44018) 2024-03-08T06:04:47.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.534+0000 7f6e63e00640 1 -- 172.21.15.40:0/3822917446 learned_addr learned my addr 172.21.15.40:0/3822917446 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:47.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.534+0000 7f6e5be00640 1 --2- 172.21.15.40:0/3822917446 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6e640b97d0 0x7f6e640ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.534+0000 7f6e63e00640 1 -- 172.21.15.40:0/3822917446 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6e640b97d0 msgr2=0x7f6e640ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:47.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.534+0000 7f6e63e00640 1 --2- 172.21.15.40:0/3822917446 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6e640b97d0 0x7f6e640ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.534+0000 7f6e63e00640 1 -- 172.21.15.40:0/3822917446 >> v1:172.21.15.53:6789/0 conn(0x7f6e6415e230 legacy=0x7f6e640b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:47.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.534+0000 7f6e63e00640 1 -- 172.21.15.40:0/3822917446 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64162800 con 0x7f6e6415de50 2024-03-08T06:04:47.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.535+0000 7f6e63e00640 1 --2- 172.21.15.40:0/3822917446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6415de50 0x7f6e64161bd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009870 tx=0x7f6e5402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d33f75dc8bcf54 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:47.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.536+0000 7f6e63400640 1 -- 172.21.15.40:0/3822917446 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54035020 con 0x7f6e6415de50 2024-03-08T06:04:47.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.536+0000 7f6e63400640 1 -- 172.21.15.40:0/3822917446 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e54031640 con 0x7f6e6415de50 2024-03-08T06:04:47.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.536+0000 7f6e63400640 1 -- 172.21.15.40:0/3822917446 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5402d040 con 0x7f6e6415de50 2024-03-08T06:04:47.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.536+0000 7f6e6aa00640 1 -- 172.21.15.40:0/3822917446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6415de50 msgr2=0x7f6e64161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.536+0000 7f6e6aa00640 1 --2- 172.21.15.40:0/3822917446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6415de50 0x7f6e64161bd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009870 tx=0x7f6e5402b8f0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.537+0000 7f6e6aa00640 1 -- 172.21.15.40:0/3822917446 shutdown_connections 2024-03-08T06:04:47.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.537+0000 7f6e6aa00640 1 --2- 172.21.15.40:0/3822917446 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6e640b97d0 0x7f6e640ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.537+0000 7f6e6aa00640 1 --2- 172.21.15.40:0/3822917446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6415de50 0x7f6e64161bd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.537+0000 7f6e6aa00640 1 -- 172.21.15.40:0/3822917446 >> 172.21.15.40:0/3822917446 conn(0x7f6e6406c490 msgr2=0x7f6e6406c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:47.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.537+0000 7f6e6aa00640 1 -- 172.21.15.40:0/3822917446 shutdown_connections 2024-03-08T06:04:47.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.537+0000 7f6e6aa00640 1 -- 172.21.15.40:0/3822917446 wait complete. 2024-03-08T06:04:47.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.538+0000 7f6e6aa00640 1 Processor -- start 2024-03-08T06:04:47.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.538+0000 7f6e6aa00640 1 -- start start 2024-03-08T06:04:47.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.539+0000 7f6e6aa00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6e640b97d0 0x7f6e64153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.539+0000 7f6e6aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e6415de50 0x7f6e640ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.539+0000 7f6e6aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6415e230 0x7f6e640ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.539+0000 7f6e6aa00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e64162290 con 0x7f6e6415e230 2024-03-08T06:04:47.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.539+0000 7f6e6aa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e64162110 con 0x7f6e6415de50 2024-03-08T06:04:47.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.539+0000 7f6e6aa00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640d3800 con 0x7f6e640b97d0 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.540+0000 7f6e68a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6415e230 0x7f6e640ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.540+0000 7f6e68a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6415e230 0x7f6e640ad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44020/0 (socket says 172.21.15.40:44020) 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.540+0000 7f6e68a00640 1 -- 172.21.15.40:0/2048249110 learned_addr learned my addr 172.21.15.40:0/2048249110 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.540+0000 7f6e5be00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e6415de50 0x7f6e640ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.540+0000 7f6e63e00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6e640b97d0 0x7f6e64153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.540+0000 7f6e68a00640 1 -- 172.21.15.40:0/2048249110 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6e640b97d0 msgr2=0x7f6e64153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.540+0000 7f6e68a00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6e640b97d0 0x7f6e64153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.540+0000 7f6e68a00640 1 -- 172.21.15.40:0/2048249110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e6415de50 msgr2=0x7f6e640ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.540+0000 7f6e68a00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e6415de50 0x7f6e640ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.540+0000 7f6e68a00640 1 -- 172.21.15.40:0/2048249110 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640b5b20 con 0x7f6e6415e230 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.540+0000 7f6e5be00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e6415de50 0x7f6e640ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.541+0000 7f6e68a00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6415e230 0x7f6e640ad760 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00b9a0 tx=0x7f6e5c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.541+0000 7f6e60c00640 1 -- 172.21.15.40:0/2048249110 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c019070 con 0x7f6e6415e230 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.541+0000 7f6e60c00640 1 -- 172.21.15.40:0/2048249110 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c00cba0 con 0x7f6e6415e230 2024-03-08T06:04:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.541+0000 7f6e6aa00640 1 -- 172.21.15.40:0/2048249110 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e640aacc0 con 0x7f6e6415e230 2024-03-08T06:04:47.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.541+0000 7f6e60c00640 1 -- 172.21.15.40:0/2048249110 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c014440 con 0x7f6e6415e230 2024-03-08T06:04:47.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.541+0000 7f6e6aa00640 1 -- 172.21.15.40:0/2048249110 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e640ab280 con 0x7f6e6415e230 2024-03-08T06:04:47.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.542+0000 7f6e6aa00640 1 -- 172.21.15.40:0/2048249110 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e640aaad0 con 0x7f6e6415e230 2024-03-08T06:04:47.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.543+0000 7f6e60c00640 1 -- 172.21.15.40:0/2048249110 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e5c01c070 con 0x7f6e6415e230 2024-03-08T06:04:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.544+0000 7f6e60c00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6e4006b4d0 0x7f6e4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.544+0000 7f6e60c00640 1 -- 172.21.15.40:0/2048249110 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e5c089e30 con 0x7f6e6415e230 2024-03-08T06:04:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.544+0000 7f6e63e00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6e4006b4d0 0x7f6e4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.545+0000 7f6e63e00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6e4006b4d0 0x7f6e4006d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e54036000 tx=0x7f6e54008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:47.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.549+0000 7f6e60c00640 1 -- 172.21.15.40:0/2048249110 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e5c058ca0 con 0x7f6e6415e230 2024-03-08T06:04:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.749+0000 7f6e6aa00640 1 -- 172.21.15.40:0/2048249110 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e64067cb0 con 0x7f6e6415e230 2024-03-08T06:04:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.750+0000 7f6e60c00640 1 -- 172.21.15.40:0/2048249110 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e5c0585a0 con 0x7f6e6415e230 2024-03-08T06:04:47.751 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 -- 172.21.15.40:0/2048249110 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6e4006b4d0 msgr2=0x7f6e4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6e4006b4d0 0x7f6e4006d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e54036000 tx=0x7f6e54008030 comp rx=0 tx=0).stop 2024-03-08T06:04:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 -- 172.21.15.40:0/2048249110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6415e230 msgr2=0x7f6e640ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6415e230 0x7f6e640ad760 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00b9a0 tx=0x7f6e5c00be60 comp rx=0 tx=0).stop 2024-03-08T06:04:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 -- 172.21.15.40:0/2048249110 shutdown_connections 2024-03-08T06:04:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6e4006b4d0 0x7f6e4006d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e6415e230 0x7f6e640ad760 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e6415de50 0x7f6e640ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 --2- 172.21.15.40:0/2048249110 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6e640b97d0 0x7f6e64153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 -- 172.21.15.40:0/2048249110 >> 172.21.15.40:0/2048249110 conn(0x7f6e6406c490 msgr2=0x7f6e6415e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 -- 172.21.15.40:0/2048249110 shutdown_connections 2024-03-08T06:04:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.753+0000 7f6e6aa00640 1 -- 172.21.15.40:0/2048249110 wait complete. 2024-03-08T06:04:47.773 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:47.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.983+0000 7f949f200640 1 Processor -- start 2024-03-08T06:04:47.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.983+0000 7f949f200640 1 -- start start 2024-03-08T06:04:47.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.985+0000 7f949f200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94a015bfb0 0x7f94a00716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.985+0000 7f949f200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94a0071c20 0x7f94a00cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.985+0000 7f949f200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a015c460 con 0x7f94a0145500 2024-03-08T06:04:47.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.985+0000 7f949f200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a0072050 con 0x7f94a015bfb0 2024-03-08T06:04:47.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.985+0000 7f949f200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a015c740 con 0x7f94a0071c20 2024-03-08T06:04:47.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.985+0000 7f949d400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94a015bfb0 0x7f94a00716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.985+0000 7f949d400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94a015bfb0 0x7f94a00716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44034/0 (socket says 172.21.15.40:44034) 2024-03-08T06:04:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.985+0000 7f949d400640 1 -- 172.21.15.40:0/4177397994 learned_addr learned my addr 172.21.15.40:0/4177397994 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.986+0000 7f949d400640 1 -- 172.21.15.40:0/4177397994 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94a0071c20 msgr2=0x7f94a00cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.986+0000 7f949e800640 1 --2- 172.21.15.40:0/4177397994 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94a0071c20 0x7f94a00cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.986+0000 7f949d400640 1 --2- 172.21.15.40:0/4177397994 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94a0071c20 0x7f94a00cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.986+0000 7f949d400640 1 -- 172.21.15.40:0/4177397994 >> v1:172.21.15.40:6789/0 conn(0x7f94a0145500 legacy=0x7f94a006ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.986+0000 7f949d400640 1 -- 172.21.15.40:0/4177397994 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a015ca20 con 0x7f94a015bfb0 2024-03-08T06:04:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.986+0000 7f949e800640 1 --2- 172.21.15.40:0/4177397994 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94a0071c20 0x7f94a00cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:47.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.987+0000 7f949d400640 1 --2- 172.21.15.40:0/4177397994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a015bfb0 0x7f94a00716e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f948c009870 tx=0x7f948c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4a67a1ec8ca71b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:47.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.987+0000 7f949ca00640 1 -- 172.21.15.40:0/4177397994 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c035020 con 0x7f94a015bfb0 2024-03-08T06:04:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.987+0000 7f949ca00640 1 -- 172.21.15.40:0/4177397994 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c031640 con 0x7f94a015bfb0 2024-03-08T06:04:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.987+0000 7f949ca00640 1 -- 172.21.15.40:0/4177397994 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c02d040 con 0x7f94a015bfb0 2024-03-08T06:04:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.988+0000 7f949f200640 1 -- 172.21.15.40:0/4177397994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a015bfb0 msgr2=0x7f94a00716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.988+0000 7f949f200640 1 --2- 172.21.15.40:0/4177397994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a015bfb0 0x7f94a00716e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f948c009870 tx=0x7f948c02b8f0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.988+0000 7f949f200640 1 -- 172.21.15.40:0/4177397994 shutdown_connections 2024-03-08T06:04:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.988+0000 7f949f200640 1 --2- 172.21.15.40:0/4177397994 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94a0071c20 0x7f94a00cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.988+0000 7f949f200640 1 --2- 172.21.15.40:0/4177397994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a015bfb0 0x7f94a00716e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.988+0000 7f949f200640 1 -- 172.21.15.40:0/4177397994 >> 172.21.15.40:0/4177397994 conn(0x7f94a00b1850 msgr2=0x7f94a00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.988+0000 7f949f200640 1 -- 172.21.15.40:0/4177397994 shutdown_connections 2024-03-08T06:04:47.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.988+0000 7f949f200640 1 -- 172.21.15.40:0/4177397994 wait complete. 2024-03-08T06:04:47.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.990+0000 7f949f200640 1 Processor -- start 2024-03-08T06:04:47.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.990+0000 7f949f200640 1 -- start start 2024-03-08T06:04:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.991+0000 7f949f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94a0071c20 0x7f94a0159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.991+0000 7f949f200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a0145500 0x7f94a013d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.991+0000 7f949f200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94a015bfb0 0x7f94a013de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.991+0000 7f949f200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00ccbc0 con 0x7f94a0145500 2024-03-08T06:04:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.991+0000 7f949f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00cca40 con 0x7f94a0071c20 2024-03-08T06:04:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.991+0000 7f949f200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00ccd40 con 0x7f94a015bfb0 2024-03-08T06:04:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.991+0000 7f949d400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a0145500 0x7f94a013d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.991+0000 7f949d400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a0145500 0x7f94a013d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44044/0 (socket says 172.21.15.40:44044) 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.991+0000 7f949d400640 1 -- 172.21.15.40:0/2466197931 learned_addr learned my addr 172.21.15.40:0/2466197931 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f949e800640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94a015bfb0 0x7f94a013de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f949de00640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94a0071c20 0x7f94a0159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f949d400640 1 -- 172.21.15.40:0/2466197931 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94a015bfb0 msgr2=0x7f94a013de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f949d400640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94a015bfb0 0x7f94a013de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f949d400640 1 -- 172.21.15.40:0/2466197931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94a0071c20 msgr2=0x7f94a0159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f949d400640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94a0071c20 0x7f94a0159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f949d400640 1 -- 172.21.15.40:0/2466197931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0159ca0 con 0x7f94a0145500 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f949de00640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94a0071c20 0x7f94a0159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f949d400640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a0145500 0x7f94a013d000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f948c02b930 tx=0x7f948c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f9496000640 1 -- 172.21.15.40:0/2466197931 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c035040 con 0x7f94a0145500 2024-03-08T06:04:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f9496000640 1 -- 172.21.15.40:0/2466197931 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c039bf0 con 0x7f94a0145500 2024-03-08T06:04:47.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.992+0000 7f9496000640 1 -- 172.21.15.40:0/2466197931 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c045480 con 0x7f94a0145500 2024-03-08T06:04:47.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.993+0000 7f949f200640 1 -- 172.21.15.40:0/2466197931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a0134bc0 con 0x7f94a0145500 2024-03-08T06:04:47.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.993+0000 7f949f200640 1 -- 172.21.15.40:0/2466197931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a0135180 con 0x7f94a0145500 2024-03-08T06:04:47.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.994+0000 7f949f200640 1 -- 172.21.15.40:0/2466197931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a0075090 con 0x7f94a0145500 2024-03-08T06:04:47.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.995+0000 7f9496000640 1 -- 172.21.15.40:0/2466197931 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f948c038020 con 0x7f94a0145500 2024-03-08T06:04:47.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.996+0000 7f9496000640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f948006b520 0x7f948006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:47.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.996+0000 7f9496000640 1 -- 172.21.15.40:0/2466197931 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f948c0ac170 con 0x7f94a0145500 2024-03-08T06:04:47.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.996+0000 7f949de00640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f948006b520 0x7f948006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:47.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:47.997+0000 7f949de00640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f948006b520 0x7f948006d9e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9490001680 tx=0x7f9490009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:48.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.000+0000 7f9496000640 1 -- 172.21.15.40:0/2466197931 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f948c031610 con 0x7f94a0145500 2024-03-08T06:04:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.200+0000 7f949f200640 1 -- 172.21.15.40:0/2466197931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94a0144130 con 0x7f94a0145500 2024-03-08T06:04:48.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.201+0000 7f9496000640 1 -- 172.21.15.40:0/2466197931 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f948c07afe0 con 0x7f94a0145500 2024-03-08T06:04:48.202 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:48.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 -- 172.21.15.40:0/2466197931 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f948006b520 msgr2=0x7f948006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:48.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f948006b520 0x7f948006d9e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9490001680 tx=0x7f9490009040 comp rx=0 tx=0).stop 2024-03-08T06:04:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 -- 172.21.15.40:0/2466197931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a0145500 msgr2=0x7f94a013d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a0145500 0x7f94a013d000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f948c02b930 tx=0x7f948c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 -- 172.21.15.40:0/2466197931 shutdown_connections 2024-03-08T06:04:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f948006b520 0x7f948006d9e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94a015bfb0 0x7f94a013de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a0145500 0x7f94a013d000 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 --2- 172.21.15.40:0/2466197931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94a0071c20 0x7f94a0159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 -- 172.21.15.40:0/2466197931 >> 172.21.15.40:0/2466197931 conn(0x7f94a00b1850 msgr2=0x7f94a015b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 -- 172.21.15.40:0/2466197931 shutdown_connections 2024-03-08T06:04:48.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.204+0000 7f949f200640 1 -- 172.21.15.40:0/2466197931 wait complete. 2024-03-08T06:04:48.224 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:04:48.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.427+0000 7f1c93e00640 1 Processor -- start 2024-03-08T06:04:48.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.428+0000 7f1c93e00640 1 -- start start 2024-03-08T06:04:48.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.429+0000 7f1c93e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940b27a0 0x7f1c940d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:48.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.429+0000 7f1c93e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c940d3410 con 0x7f1c941611f0 2024-03-08T06:04:48.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.429+0000 7f1c93e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c940d36f0 con 0x7f1c940b5bf0 2024-03-08T06:04:48.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.429+0000 7f1c93e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940ceac0 con 0x7f1c940b27a0 2024-03-08T06:04:48.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.430+0000 7f1c92a00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1c941611f0 0x7f1c940b54e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40098/0 (socket says 172.21.15.40:40098) 2024-03-08T06:04:48.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.430+0000 7f1c92a00640 1 -- 172.21.15.40:0/2056255105 learned_addr learned my addr 172.21.15.40:0/2056255105 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:48.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.430+0000 7f1c93400640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940b27a0 0x7f1c940d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:48.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.430+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558022781 0 0) 0x7f1c940d3410 con 0x7f1c941611f0 2024-03-08T06:04:48.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.430+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c74003520 con 0x7f1c941611f0 2024-03-08T06:04:48.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 452607216 0 0) 0x7f1c74003520 con 0x7f1c941611f0 2024-03-08T06:04:48.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940b27a0 msgr2=0x7f1c940d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940b27a0 0x7f1c940d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 >> v1:172.21.15.53:6789/0 conn(0x7f1c940b5bf0 legacy=0x7f1c940d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c940ced10 con 0x7f1c941611f0 2024-03-08T06:04:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f1c84001670 con 0x7f1c941611f0 2024-03-08T06:04:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 >> v1:172.21.15.40:6789/0 conn(0x7f1c941611f0 legacy=0x7f1c940b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940b27a0 0x7f1c74003ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c74004040 0x7f1c74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c74004db0 0x7f1c74005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940ceac0 con 0x7f1c74004040 2024-03-08T06:04:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7400bcc0 con 0x7f1c74004db0 2024-03-08T06:04:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7400bec0 con 0x7f1c940b27a0 2024-03-08T06:04:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.431+0000 7f1c92000640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c74004040 0x7f1c74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.432+0000 7f1c92a00640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940b27a0 0x7f1c74003ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.432+0000 7f1c93400640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c74004db0 0x7f1c74005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.432+0000 7f1c92000640 1 -- 172.21.15.40:0/2056255105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940b27a0 msgr2=0x7f1c74003ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.432+0000 7f1c92000640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940b27a0 0x7f1c74003ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.432+0000 7f1c92000640 1 -- 172.21.15.40:0/2056255105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c74004db0 msgr2=0x7f1c74005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.432+0000 7f1c92000640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c74004db0 0x7f1c74005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.432+0000 7f1c92000640 1 -- 172.21.15.40:0/2056255105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c74003520 con 0x7f1c74004040 2024-03-08T06:04:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.433+0000 7f1c92000640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c74004040 0x7f1c74004800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c001e50 tx=0x7f1c74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6097858df525aeb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.433+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c001a70 con 0x7f1c74004040 2024-03-08T06:04:48.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.433+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7c003b70 con 0x7f1c74004040 2024-03-08T06:04:48.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.433+0000 7f1c91600640 1 -- 172.21.15.40:0/2056255105 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0050d0 con 0x7f1c74004040 2024-03-08T06:04:48.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.434+0000 7f1c93e00640 1 -- 172.21.15.40:0/2056255105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c74004040 msgr2=0x7f1c74004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:48.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.434+0000 7f1c93e00640 1 --2- 172.21.15.40:0/2056255105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c74004040 0x7f1c74004800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c001e50 tx=0x7f1c74005830 comp rx=0 tx=0).stop 2024-03-08T06:04:48.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.434+0000 7f1c93400640 1 -- 172.21.15.40:0/2056255105 reap_dead start 2024-03-08T06:04:48.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.434+0000 7f1c93e00640 1 -- 172.21.15.40:0/2056255105 shutdown_connections 2024-03-08T06:04:48.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.434+0000 7f1c93e00640 1 -- 172.21.15.40:0/2056255105 >> 172.21.15.40:0/2056255105 conn(0x7f1c9406c700 msgr2=0x7f1c9406cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:48.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.434+0000 7f1c93e00640 1 -- 172.21.15.40:0/2056255105 shutdown_connections 2024-03-08T06:04:48.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.434+0000 7f1c93e00640 1 -- 172.21.15.40:0/2056255105 wait complete. 2024-03-08T06:04:48.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.436+0000 7f1c93e00640 1 Processor -- start 2024-03-08T06:04:48.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.437+0000 7f1c93e00640 1 -- start start 2024-03-08T06:04:48.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.438+0000 7f1c93e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c74004040 0x7f1c941e7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:48.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.438+0000 7f1c93e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940cb790 0x7f1c94155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.438+0000 7f1c93e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c941f7550 0x7f1c941f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.438+0000 7f1c93e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940cf080 con 0x7f1c941f7550 2024-03-08T06:04:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.438+0000 7f1c93e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940d0ff0 con 0x7f1c74004040 2024-03-08T06:04:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.438+0000 7f1c93e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940d1170 con 0x7f1c940cb790 2024-03-08T06:04:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.438+0000 7f1c93400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c941f7550 0x7f1c941f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.438+0000 7f1c93400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c941f7550 0x7f1c941f7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44066/0 (socket says 172.21.15.40:44066) 2024-03-08T06:04:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.438+0000 7f1c93400640 1 -- 172.21.15.40:0/3329098676 learned_addr learned my addr 172.21.15.40:0/3329098676 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.438+0000 7f1c92a00640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c74004040 0x7f1c941e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.438+0000 7f1c92000640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940cb790 0x7f1c94155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.439+0000 7f1c93400640 1 -- 172.21.15.40:0/3329098676 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940cb790 msgr2=0x7f1c94155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.439+0000 7f1c93400640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940cb790 0x7f1c94155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.439+0000 7f1c93400640 1 -- 172.21.15.40:0/3329098676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c74004040 msgr2=0x7f1c941e7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.439+0000 7f1c93400640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c74004040 0x7f1c941e7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.439+0000 7f1c93400640 1 -- 172.21.15.40:0/3329098676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c940cbc70 con 0x7f1c941f7550 2024-03-08T06:04:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.439+0000 7f1c92a00640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c74004040 0x7f1c941e7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.439+0000 7f1c93400640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c941f7550 0x7f1c941f7a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1c940d0a00 tx=0x7f1c8c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.439+0000 7f1c8b400640 1 -- 172.21.15.40:0/3329098676 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c019020 con 0x7f1c941f7550 2024-03-08T06:04:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.439+0000 7f1c8b400640 1 -- 172.21.15.40:0/3329098676 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c003610 con 0x7f1c941f7550 2024-03-08T06:04:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.439+0000 7f1c8b400640 1 -- 172.21.15.40:0/3329098676 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c003a00 con 0x7f1c941f7550 2024-03-08T06:04:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.439+0000 7f1c93e00640 1 -- 172.21.15.40:0/3329098676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c940cbf50 con 0x7f1c941f7550 2024-03-08T06:04:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.440+0000 7f1c93e00640 1 -- 172.21.15.40:0/3329098676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c940cc510 con 0x7f1c941f7550 2024-03-08T06:04:48.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.441+0000 7f1c93e00640 1 -- 172.21.15.40:0/3329098676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c94157a70 con 0x7f1c941f7550 2024-03-08T06:04:48.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.442+0000 7f1c8b400640 1 -- 172.21.15.40:0/3329098676 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c8c01c020 con 0x7f1c941f7550 2024-03-08T06:04:48.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.443+0000 7f1c8b400640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c7006b4d0 0x7f1c7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.443+0000 7f1c8b400640 1 -- 172.21.15.40:0/3329098676 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1c8c08a470 con 0x7f1c941f7550 2024-03-08T06:04:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.443+0000 7f1c92a00640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c7006b4d0 0x7f1c7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.444+0000 7f1c92a00640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c7006b4d0 0x7f1c7006d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1c84001eb0 tx=0x7f1c84009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:48.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.447+0000 7f1c8b400640 1 -- 172.21.15.40:0/3329098676 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c8c023030 con 0x7f1c941f7550 2024-03-08T06:04:48.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.659+0000 7f1c93e00640 1 -- 172.21.15.40:0/3329098676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1c94075090 con 0x7f1c941f7550 2024-03-08T06:04:48.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.660+0000 7f1c8b400640 1 -- 172.21.15.40:0/3329098676 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1c8c059300 con 0x7f1c941f7550 2024-03-08T06:04:48.661 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-03-08T06:04:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.662+0000 7f1c93e00640 1 -- 172.21.15.40:0/3329098676 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c7006b4d0 msgr2=0x7f1c7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.662+0000 7f1c93e00640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c7006b4d0 0x7f1c7006d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1c84001eb0 tx=0x7f1c84009040 comp rx=0 tx=0).stop 2024-03-08T06:04:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.662+0000 7f1c93e00640 1 -- 172.21.15.40:0/3329098676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c941f7550 msgr2=0x7f1c941f7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.662+0000 7f1c93e00640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c941f7550 0x7f1c941f7a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1c940d0a00 tx=0x7f1c8c011920 comp rx=0 tx=0).stop 2024-03-08T06:04:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.662+0000 7f1c93e00640 1 -- 172.21.15.40:0/3329098676 shutdown_connections 2024-03-08T06:04:48.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.663+0000 7f1c93e00640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c7006b4d0 0x7f1c7006d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.663+0000 7f1c93e00640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c941f7550 0x7f1c941f7a00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.663+0000 7f1c93e00640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1c940cb790 0x7f1c94155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.663+0000 7f1c93e00640 1 --2- 172.21.15.40:0/3329098676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c74004040 0x7f1c941e7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:48.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.663+0000 7f1c93e00640 1 -- 172.21.15.40:0/3329098676 >> 172.21.15.40:0/3329098676 conn(0x7f1c9406c700 msgr2=0x7f1c940b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:48.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.663+0000 7f1c93e00640 1 -- 172.21.15.40:0/3329098676 shutdown_connections 2024-03-08T06:04:48.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:48.663+0000 7f1c93e00640 1 -- 172.21.15.40:0/3329098676 wait complete. 2024-03-08T06:04:48.683 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-08T06:04:48.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:48.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-08T06:04:48.797 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:04:48.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:48.797 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-08T06:04:48.849 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:04:48.849 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:48.909 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:04:48.909 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:48.966 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:04:48.966 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:49.018 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:49.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.227+0000 7fdbf6a00640 1 Processor -- start 2024-03-08T06:04:49.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.227+0000 7fdbf6a00640 1 -- start start 2024-03-08T06:04:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.229+0000 7fdbf6a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdbf0145520 0x7fdbf015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.229+0000 7fdbf6a00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf015ceb0 con 0x7fdbf015bea0 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.229+0000 7fdbf6a00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf015c330 con 0x7fdbf0071df0 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.229+0000 7fdbf6a00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf015c610 con 0x7fdbf0145520 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.229+0000 7fdbf4c00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdbf015bea0 0x7fdbf00716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40114/0 (socket says 172.21.15.40:40114) 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.229+0000 7fdbf4c00640 1 -- 172.21.15.40:0/98343134 learned_addr learned my addr 172.21.15.40:0/98343134 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.230+0000 7fdbf5600640 1 --2- 172.21.15.40:0/98343134 >> v2:172.21.15.53:3300/0 conn(0x7fdbf0145520 0x7fdbf015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.230+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462242588 0 0) 0x7fdbf015ceb0 con 0x7fdbf015bea0 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.230+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd8003520 con 0x7fdbf015bea0 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.230+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2379055308 0 0) 0x7fdbd8003520 con 0x7fdbf015bea0 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.230+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 >> v2:172.21.15.53:3300/0 conn(0x7fdbf0145520 msgr2=0x7fdbf015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.230+0000 7fdbefe00640 1 --2- 172.21.15.40:0/98343134 >> v2:172.21.15.53:3300/0 conn(0x7fdbf0145520 0x7fdbf015d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.230+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 >> v1:172.21.15.53:6789/0 conn(0x7fdbf0071df0 legacy=0x7fdbf006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.230+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf015c8f0 con 0x7fdbf015bea0 2024-03-08T06:04:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.230+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fdbdc001670 con 0x7fdbf015bea0 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.230+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 >> v1:172.21.15.40:6789/0 conn(0x7fdbf015bea0 legacy=0x7fdbf00716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbefe00640 1 --2- 172.21.15.40:0/98343134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd80039b0 0x7fdbd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbefe00640 1 --2- 172.21.15.40:0/98343134 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdbd8004800 0x7fdbd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbefe00640 1 --2- 172.21.15.40:0/98343134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd80051c0 0x7fdbd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf015c610 con 0x7fdbd80039b0 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd800c060 con 0x7fdbd80051c0 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd800c260 con 0x7fdbd8004800 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbf5600640 1 --2- 172.21.15.40:0/98343134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd80039b0 0x7fdbd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbf5600640 1 -- 172.21.15.40:0/98343134 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdbd8004800 msgr2=0x7fdbd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbf5600640 1 --2- 172.21.15.40:0/98343134 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdbd8004800 0x7fdbd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbf5600640 1 -- 172.21.15.40:0/98343134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd80051c0 msgr2=0x7fdbd800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbf6000640 1 --2- 172.21.15.40:0/98343134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd80051c0 0x7fdbd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbf5600640 1 --2- 172.21.15.40:0/98343134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd80051c0 0x7fdbd800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbf6000640 1 -- 172.21.15.40:0/98343134 reap_dead start 2024-03-08T06:04:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.231+0000 7fdbf5600640 1 -- 172.21.15.40:0/98343134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd8003520 con 0x7fdbd80039b0 2024-03-08T06:04:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.233+0000 7fdbf5600640 1 --2- 172.21.15.40:0/98343134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd80039b0 0x7fdbd8003dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009ac0 tx=0x7fdbd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8027dd0df1a61d66 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.233+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4039070 con 0x7fdbd80039b0 2024-03-08T06:04:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.233+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe402c6b0 con 0x7fdbd80039b0 2024-03-08T06:04:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.233+0000 7fdbefe00640 1 -- 172.21.15.40:0/98343134 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4009690 con 0x7fdbd80039b0 2024-03-08T06:04:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.233+0000 7fdbf6a00640 1 -- 172.21.15.40:0/98343134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd80039b0 msgr2=0x7fdbd8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.233+0000 7fdbf6a00640 1 --2- 172.21.15.40:0/98343134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd80039b0 0x7fdbd8003dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009ac0 tx=0x7fdbd8005830 comp rx=0 tx=0).stop 2024-03-08T06:04:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.234+0000 7fdbf6a00640 1 -- 172.21.15.40:0/98343134 shutdown_connections 2024-03-08T06:04:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.234+0000 7fdbf6a00640 1 --2- 172.21.15.40:0/98343134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd80039b0 0x7fdbd8003dd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.234+0000 7fdbf6a00640 1 -- 172.21.15.40:0/98343134 >> 172.21.15.40:0/98343134 conn(0x7fdbf00b1850 msgr2=0x7fdbf00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.234+0000 7fdbf6a00640 1 -- 172.21.15.40:0/98343134 shutdown_connections 2024-03-08T06:04:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.234+0000 7fdbf6a00640 1 -- 172.21.15.40:0/98343134 wait complete. 2024-03-08T06:04:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.235+0000 7fdbf6a00640 1 Processor -- start 2024-03-08T06:04:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.236+0000 7fdbf6a00640 1 -- start start 2024-03-08T06:04:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.236+0000 7fdbf6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd80039b0 0x7fdbf014f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.236+0000 7fdbf6a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdbf014f640 0x7fdbf014fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00acbf0 0x7fdbf00ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf00af1c0 con 0x7fdbf00acbf0 2024-03-08T06:04:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf00af040 con 0x7fdbd80039b0 2024-03-08T06:04:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf006e410 con 0x7fdbf014f640 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00acbf0 0x7fdbf00ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00acbf0 0x7fdbf00ad070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44080/0 (socket says 172.21.15.40:44080) 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6000640 1 -- 172.21.15.40:0/2019769967 learned_addr learned my addr 172.21.15.40:0/2019769967 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf5600640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd80039b0 0x7fdbf014f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf4c00640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdbf014f640 0x7fdbf014fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6000640 1 -- 172.21.15.40:0/2019769967 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdbf014f640 msgr2=0x7fdbf014fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6000640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdbf014f640 0x7fdbf014fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6000640 1 -- 172.21.15.40:0/2019769967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd80039b0 msgr2=0x7fdbf014f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6000640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd80039b0 0x7fdbf014f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf6000640 1 -- 172.21.15.40:0/2019769967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf00b49b0 con 0x7fdbf00acbf0 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.237+0000 7fdbf4c00640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdbf014f640 0x7fdbf014fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.238+0000 7fdbf6000640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00acbf0 0x7fdbf00ad070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00cd380 tx=0x7fdbe800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.238+0000 7fdbed600640 1 -- 172.21.15.40:0/2019769967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8018070 con 0x7fdbf00acbf0 2024-03-08T06:04:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.238+0000 7fdbed600640 1 -- 172.21.15.40:0/2019769967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe8003d80 con 0x7fdbf00acbf0 2024-03-08T06:04:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.238+0000 7fdbf6a00640 1 -- 172.21.15.40:0/2019769967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf00b3910 con 0x7fdbf00acbf0 2024-03-08T06:04:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.238+0000 7fdbed600640 1 -- 172.21.15.40:0/2019769967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8005270 con 0x7fdbf00acbf0 2024-03-08T06:04:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.238+0000 7fdbf6a00640 1 -- 172.21.15.40:0/2019769967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf00b3ed0 con 0x7fdbf00acbf0 2024-03-08T06:04:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.240+0000 7fdbf6a00640 1 -- 172.21.15.40:0/2019769967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf0075100 con 0x7fdbf00acbf0 2024-03-08T06:04:49.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.240+0000 7fdbed600640 1 -- 172.21.15.40:0/2019769967 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdbe801b070 con 0x7fdbf00acbf0 2024-03-08T06:04:49.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.242+0000 7fdbed600640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdbd006b500 0x7fdbd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.242+0000 7fdbed600640 1 -- 172.21.15.40:0/2019769967 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdbe8089750 con 0x7fdbf00acbf0 2024-03-08T06:04:49.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.242+0000 7fdbf5600640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdbd006b500 0x7fdbd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.243+0000 7fdbf5600640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdbd006b500 0x7fdbd006d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4036000 tx=0x7fdbe4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:49.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.247+0000 7fdbed600640 1 -- 172.21.15.40:0/2019769967 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdbe8057e60 con 0x7fdbf00acbf0 2024-03-08T06:04:49.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.447+0000 7fdbf6a00640 1 -- 172.21.15.40:0/2019769967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbf0071a30 con 0x7fdbf00acbf0 2024-03-08T06:04:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.450+0000 7fdbed600640 1 -- 172.21.15.40:0/2019769967 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16770 (secure 0 0 0) 0x7fdbe805ce20 con 0x7fdbf00acbf0 2024-03-08T06:04:49.452 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:49.452 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:48.785806+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:49.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.453+0000 7fdbf6a00640 1 -- 172.21.15.40:0/2019769967 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdbd006b500 msgr2=0x7fdbd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.453+0000 7fdbf6a00640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdbd006b500 0x7fdbd006d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4036000 tx=0x7fdbe4030000 comp rx=0 tx=0).stop 2024-03-08T06:04:49.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.453+0000 7fdbf6a00640 1 -- 172.21.15.40:0/2019769967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00acbf0 msgr2=0x7fdbf00ad070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.453+0000 7fdbf6a00640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00acbf0 0x7fdbf00ad070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00cd380 tx=0x7fdbe800d9d0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.453+0000 7fdbf6a00640 1 -- 172.21.15.40:0/2019769967 shutdown_connections 2024-03-08T06:04:49.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.453+0000 7fdbf6a00640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fdbd006b500 0x7fdbd006d9c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.453+0000 7fdbf6a00640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00acbf0 0x7fdbf00ad070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.453+0000 7fdbf6a00640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdbf014f640 0x7fdbf014fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.453+0000 7fdbf6a00640 1 --2- 172.21.15.40:0/2019769967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd80039b0 0x7fdbf014f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.453+0000 7fdbf6a00640 1 -- 172.21.15.40:0/2019769967 >> 172.21.15.40:0/2019769967 conn(0x7fdbf00b1850 msgr2=0x7fdbf015e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:49.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.454+0000 7fdbf6a00640 1 -- 172.21.15.40:0/2019769967 shutdown_connections 2024-03-08T06:04:49.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.454+0000 7fdbf6a00640 1 -- 172.21.15.40:0/2019769967 wait complete. 2024-03-08T06:04:49.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:49.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:49.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.702+0000 7f2c7e400640 1 Processor -- start 2024-03-08T06:04:49.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.702+0000 7f2c7e400640 1 -- start start 2024-03-08T06:04:49.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.703+0000 7f2c7e400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c7815e240 0x7f2c780d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.703+0000 7f2c7e400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c780b5f00 0x7f2c780cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.703+0000 7f2c7e400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c780b6e00 con 0x7f2c7815de60 2024-03-08T06:04:49.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.703+0000 7f2c7e400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c780d3660 con 0x7f2c780b5f00 2024-03-08T06:04:49.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.703+0000 7f2c7e400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780d24e0 con 0x7f2c7815e240 2024-03-08T06:04:49.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.704+0000 7f2c77400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2c7815de60 0x7f2c780b57f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40130/0 (socket says 172.21.15.40:40130) 2024-03-08T06:04:49.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.704+0000 7f2c77400640 1 -- 172.21.15.40:0/478507553 learned_addr learned my addr 172.21.15.40:0/478507553 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:49.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.704+0000 7f2c77e00640 1 --2- 172.21.15.40:0/478507553 >> v2:172.21.15.40:3300/0 conn(0x7f2c780b5f00 0x7f2c780cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.704+0000 7f2c76a00640 1 --2- 172.21.15.40:0/478507553 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c7815e240 0x7f2c780d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.704+0000 7f2c77e00640 1 -- 172.21.15.40:0/478507553 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c7815e240 msgr2=0x7f2c780d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:49.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.704+0000 7f2c77e00640 1 --2- 172.21.15.40:0/478507553 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c7815e240 0x7f2c780d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.704+0000 7f2c76000640 1 -- 172.21.15.40:0/478507553 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642041163 0 0) 0x7f2c780b6e00 con 0x7f2c7815de60 2024-03-08T06:04:49.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.704+0000 7f2c77e00640 1 -- 172.21.15.40:0/478507553 >> v1:172.21.15.40:6789/0 conn(0x7f2c7815de60 legacy=0x7f2c780b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.704+0000 7f2c77e00640 1 -- 172.21.15.40:0/478507553 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c780d27c0 con 0x7f2c780b5f00 2024-03-08T06:04:49.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.706+0000 7f2c77e00640 1 --2- 172.21.15.40:0/478507553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c780b5f00 0x7f2c780cefe0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2c700097a0 tx=0x7f2c7002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4943a8c0cd25d7b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:49.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.706+0000 7f2c76000640 1 -- 172.21.15.40:0/478507553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70031780 con 0x7f2c780b5f00 2024-03-08T06:04:49.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.706+0000 7f2c76000640 1 -- 172.21.15.40:0/478507553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c70037070 con 0x7f2c780b5f00 2024-03-08T06:04:49.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.706+0000 7f2c76000640 1 -- 172.21.15.40:0/478507553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70031b40 con 0x7f2c780b5f00 2024-03-08T06:04:49.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.707+0000 7f2c7e400640 1 -- 172.21.15.40:0/478507553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c780b5f00 msgr2=0x7f2c780cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.707+0000 7f2c7e400640 1 --2- 172.21.15.40:0/478507553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c780b5f00 0x7f2c780cefe0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2c700097a0 tx=0x7f2c7002b820 comp rx=0 tx=0).stop 2024-03-08T06:04:49.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.707+0000 7f2c7e400640 1 -- 172.21.15.40:0/478507553 shutdown_connections 2024-03-08T06:04:49.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.707+0000 7f2c7e400640 1 --2- 172.21.15.40:0/478507553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c780b5f00 0x7f2c780cefe0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.707+0000 7f2c7e400640 1 --2- 172.21.15.40:0/478507553 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c7815e240 0x7f2c780d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.707+0000 7f2c7e400640 1 -- 172.21.15.40:0/478507553 >> 172.21.15.40:0/478507553 conn(0x7f2c78067240 msgr2=0x7f2c780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:49.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.707+0000 7f2c7e400640 1 -- 172.21.15.40:0/478507553 shutdown_connections 2024-03-08T06:04:49.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.707+0000 7f2c7e400640 1 -- 172.21.15.40:0/478507553 wait complete. 2024-03-08T06:04:49.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.709+0000 7f2c7e400640 1 Processor -- start 2024-03-08T06:04:49.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.709+0000 7f2c7e400640 1 -- start start 2024-03-08T06:04:49.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.710+0000 7f2c7e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c780b5f00 0x7f2c780ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.710+0000 7f2c7e400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c7815de60 0x7f2c78160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.710+0000 7f2c7e400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c7815e240 0x7f2c78160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.710+0000 7f2c7e400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780b72e0 con 0x7f2c7815e240 2024-03-08T06:04:49.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.710+0000 7f2c7e400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780d2d80 con 0x7f2c780b5f00 2024-03-08T06:04:49.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.710+0000 7f2c7e400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780b7460 con 0x7f2c7815de60 2024-03-08T06:04:49.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.710+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c7815e240 0x7f2c78160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.710+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c7815e240 0x7f2c78160a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44104/0 (socket says 172.21.15.40:44104) 2024-03-08T06:04:49.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.710+0000 7f2c77e00640 1 -- 172.21.15.40:0/2054871365 learned_addr learned my addr 172.21.15.40:0/2054871365 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:49.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.710+0000 7f2c76a00640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c7815de60 0x7f2c78160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.710+0000 7f2c77400640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c780b5f00 0x7f2c780ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.711+0000 7f2c77e00640 1 -- 172.21.15.40:0/2054871365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c7815de60 msgr2=0x7f2c78160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.711+0000 7f2c77e00640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c7815de60 0x7f2c78160530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.711+0000 7f2c77e00640 1 -- 172.21.15.40:0/2054871365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c780b5f00 msgr2=0x7f2c780ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.711+0000 7f2c77e00640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c780b5f00 0x7f2c780ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.711+0000 7f2c77e00640 1 -- 172.21.15.40:0/2054871365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7814ac20 con 0x7f2c7815e240 2024-03-08T06:04:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.711+0000 7f2c77e00640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c7815e240 0x7f2c78160a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c700097a0 tx=0x7f2c700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.711+0000 7f2c6f400640 1 -- 172.21.15.40:0/2054871365 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70032610 con 0x7f2c7815e240 2024-03-08T06:04:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.711+0000 7f2c6f400640 1 -- 172.21.15.40:0/2054871365 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c700327b0 con 0x7f2c7815e240 2024-03-08T06:04:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.711+0000 7f2c6f400640 1 -- 172.21.15.40:0/2054871365 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70032b80 con 0x7f2c7815e240 2024-03-08T06:04:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.711+0000 7f2c7e400640 1 -- 172.21.15.40:0/2054871365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7814af00 con 0x7f2c7815e240 2024-03-08T06:04:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.712+0000 7f2c7e400640 1 -- 172.21.15.40:0/2054871365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7814b4c0 con 0x7f2c7815e240 2024-03-08T06:04:49.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.713+0000 7f2c7e400640 1 -- 172.21.15.40:0/2054871365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c78067cb0 con 0x7f2c7815e240 2024-03-08T06:04:49.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.714+0000 7f2c6f400640 1 -- 172.21.15.40:0/2054871365 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c700453f0 con 0x7f2c7815e240 2024-03-08T06:04:49.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.715+0000 7f2c6f400640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2c5406b4d0 0x7f2c5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:49.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.715+0000 7f2c6f400640 1 -- 172.21.15.40:0/2054871365 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2c700ac1f0 con 0x7f2c7815e240 2024-03-08T06:04:49.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.715+0000 7f2c77400640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2c5406b4d0 0x7f2c5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:49.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.715+0000 7f2c77400640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2c5406b4d0 0x7f2c5406d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000f60 tx=0x7f2c68007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:49.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.719+0000 7f2c6f400640 1 -- 172.21.15.40:0/2054871365 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c70040070 con 0x7f2c7815e240 2024-03-08T06:04:49.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.918+0000 7f2c7e400640 1 -- 172.21.15.40:0/2054871365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c780b3070 con 0x7f2c7815e240 2024-03-08T06:04:49.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.919+0000 7f2c6f400640 1 -- 172.21.15.40:0/2054871365 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c7002f050 con 0x7f2c7815e240 2024-03-08T06:04:49.920 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:49.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.921+0000 7f2c7e400640 1 -- 172.21.15.40:0/2054871365 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2c5406b4d0 msgr2=0x7f2c5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.921+0000 7f2c7e400640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2c5406b4d0 0x7f2c5406d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000f60 tx=0x7f2c68007b50 comp rx=0 tx=0).stop 2024-03-08T06:04:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.921+0000 7f2c7e400640 1 -- 172.21.15.40:0/2054871365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c7815e240 msgr2=0x7f2c78160a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.921+0000 7f2c7e400640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c7815e240 0x7f2c78160a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c700097a0 tx=0x7f2c700046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.921+0000 7f2c7e400640 1 -- 172.21.15.40:0/2054871365 shutdown_connections 2024-03-08T06:04:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.921+0000 7f2c7e400640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2c5406b4d0 0x7f2c5406d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.921+0000 7f2c7e400640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c7815e240 0x7f2c78160a70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.921+0000 7f2c7e400640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c7815de60 0x7f2c78160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.921+0000 7f2c7e400640 1 --2- 172.21.15.40:0/2054871365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c780b5f00 0x7f2c780ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.921+0000 7f2c7e400640 1 -- 172.21.15.40:0/2054871365 >> 172.21.15.40:0/2054871365 conn(0x7f2c78067240 msgr2=0x7f2c7815fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.922+0000 7f2c7e400640 1 -- 172.21.15.40:0/2054871365 shutdown_connections 2024-03-08T06:04:49.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:49.922+0000 7f2c7e400640 1 -- 172.21.15.40:0/2054871365 wait complete. 2024-03-08T06:04:49.941 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:50.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.163+0000 7efdd6e00640 1 Processor -- start 2024-03-08T06:04:50.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.163+0000 7efdd6e00640 1 -- start start 2024-03-08T06:04:50.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.165+0000 7efdd6e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efdd015bfb0 0x7efdd00716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.165+0000 7efdd6e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd015c460 con 0x7efdd0071c20 2024-03-08T06:04:50.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.165+0000 7efdd6e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd015c740 con 0x7efdd0145500 2024-03-08T06:04:50.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.165+0000 7efdd6e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd0072050 con 0x7efdd015bfb0 2024-03-08T06:04:50.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.165+0000 7efdd6400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efdd0071c20 0x7efdd00cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40138/0 (socket says 172.21.15.40:40138) 2024-03-08T06:04:50.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.165+0000 7efdd6400640 1 -- 172.21.15.40:0/3180357705 learned_addr learned my addr 172.21.15.40:0/3180357705 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdd5000640 1 --2- 172.21.15.40:0/3180357705 >> v2:172.21.15.53:3300/0 conn(0x7efdd015bfb0 0x7efdd00716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119658780 0 0) 0x7efdd015c460 con 0x7efdd0071c20 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdb8003520 con 0x7efdd0071c20 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 278436848 0 0) 0x7efdb8003520 con 0x7efdd0071c20 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 >> v2:172.21.15.53:3300/0 conn(0x7efdd015bfb0 msgr2=0x7efdd00716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 --2- 172.21.15.40:0/3180357705 >> v2:172.21.15.53:3300/0 conn(0x7efdd015bfb0 0x7efdd00716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 >> v1:172.21.15.53:6789/0 conn(0x7efdd0145500 legacy=0x7efdd006ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd015ca20 con 0x7efdd0071c20 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7efdc8001670 con 0x7efdd0071c20 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 >> v1:172.21.15.40:6789/0 conn(0x7efdd0071c20 legacy=0x7efdd00cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 --2- 172.21.15.40:0/3180357705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb8003a20 0x7efdb8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 --2- 172.21.15.40:0/3180357705 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efdb8004800 0x7efdb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 --2- 172.21.15.40:0/3180357705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 0x7efdb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd0072050 con 0x7efdb8005230 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb800c190 con 0x7efdb8003a20 2024-03-08T06:04:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.166+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb800c390 con 0x7efdb8004800 2024-03-08T06:04:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.167+0000 7efdd6400640 1 --2- 172.21.15.40:0/3180357705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 0x7efdb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.167+0000 7efdd6400640 1 -- 172.21.15.40:0/3180357705 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efdb8004800 msgr2=0x7efdb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.167+0000 7efdd6400640 1 --2- 172.21.15.40:0/3180357705 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efdb8004800 0x7efdb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.167+0000 7efdd6400640 1 -- 172.21.15.40:0/3180357705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb8003a20 msgr2=0x7efdb8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.167+0000 7efdd6400640 1 --2- 172.21.15.40:0/3180357705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb8003a20 0x7efdb8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.167+0000 7efdd6400640 1 -- 172.21.15.40:0/3180357705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb8003520 con 0x7efdb8005230 2024-03-08T06:04:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.168+0000 7efdd6400640 1 -- 172.21.15.40:0/3180357705 reap_dead start 2024-03-08T06:04:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.168+0000 7efdd6400640 1 --2- 172.21.15.40:0/3180357705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 0x7efdb800b9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efdc800c240 tx=0x7efdb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223d126e26164437 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.168+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8007cf0 con 0x7efdb8005230 2024-03-08T06:04:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.168+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc8030070 con 0x7efdb8005230 2024-03-08T06:04:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.168+0000 7efdcfe00640 1 -- 172.21.15.40:0/3180357705 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8005d30 con 0x7efdb8005230 2024-03-08T06:04:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.169+0000 7efdd6e00640 1 -- 172.21.15.40:0/3180357705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 msgr2=0x7efdb800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.169+0000 7efdd6e00640 1 --2- 172.21.15.40:0/3180357705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 0x7efdb800b9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efdc800c240 tx=0x7efdb8005830 comp rx=0 tx=0).stop 2024-03-08T06:04:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.169+0000 7efdd6e00640 1 -- 172.21.15.40:0/3180357705 shutdown_connections 2024-03-08T06:04:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.169+0000 7efdd6e00640 1 --2- 172.21.15.40:0/3180357705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 0x7efdb800b9c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.169+0000 7efdd6e00640 1 -- 172.21.15.40:0/3180357705 >> 172.21.15.40:0/3180357705 conn(0x7efdd00b1850 msgr2=0x7efdd00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.169+0000 7efdd6e00640 1 -- 172.21.15.40:0/3180357705 shutdown_connections 2024-03-08T06:04:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.169+0000 7efdd6e00640 1 -- 172.21.15.40:0/3180357705 wait complete. 2024-03-08T06:04:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.171+0000 7efdd6e00640 1 Processor -- start 2024-03-08T06:04:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.171+0000 7efdd6e00640 1 -- start start 2024-03-08T06:04:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.172+0000 7efdd6e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 0x7efdd015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.172+0000 7efdd6e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efdd01e52e0 0x7efdd01e5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.172+0000 7efdd6e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdd00b36c0 0x7efdd00b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.172+0000 7efdd6e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd0071860 con 0x7efdb8005230 2024-03-08T06:04:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.172+0000 7efdd6e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00716e0 con 0x7efdd00b36c0 2024-03-08T06:04:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.172+0000 7efdd6e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00719e0 con 0x7efdd01e52e0 2024-03-08T06:04:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.172+0000 7efdd5a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 0x7efdd015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.172+0000 7efdd5a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 0x7efdd015adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44118/0 (socket says 172.21.15.40:44118) 2024-03-08T06:04:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.172+0000 7efdd5a00640 1 -- 172.21.15.40:0/151017267 learned_addr learned my addr 172.21.15.40:0/151017267 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.173+0000 7efdd5000640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efdd01e52e0 0x7efdd01e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.173+0000 7efdd6400640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdd00b36c0 0x7efdd00b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.173+0000 7efdd5a00640 1 -- 172.21.15.40:0/151017267 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efdd01e52e0 msgr2=0x7efdd01e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.173+0000 7efdd5a00640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efdd01e52e0 0x7efdd01e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.173+0000 7efdd5a00640 1 -- 172.21.15.40:0/151017267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdd00b36c0 msgr2=0x7efdd00b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.173+0000 7efdd5a00640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdd00b36c0 0x7efdd00b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.173+0000 7efdd5a00640 1 -- 172.21.15.40:0/151017267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd01e5dc0 con 0x7efdb8005230 2024-03-08T06:04:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.173+0000 7efdd5a00640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 0x7efdd015adb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efdc400c790 tx=0x7efdc400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.173+0000 7efdcd600640 1 -- 172.21.15.40:0/151017267 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4003b70 con 0x7efdb8005230 2024-03-08T06:04:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.174+0000 7efdcd600640 1 -- 172.21.15.40:0/151017267 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc4003d30 con 0x7efdb8005230 2024-03-08T06:04:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.174+0000 7efdcd600640 1 -- 172.21.15.40:0/151017267 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc400aa90 con 0x7efdb8005230 2024-03-08T06:04:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.174+0000 7efdd6e00640 1 -- 172.21.15.40:0/151017267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd01e60a0 con 0x7efdb8005230 2024-03-08T06:04:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.174+0000 7efdd6e00640 1 -- 172.21.15.40:0/151017267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd01e6660 con 0x7efdb8005230 2024-03-08T06:04:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.175+0000 7efdd6e00640 1 -- 172.21.15.40:0/151017267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd00ad0e0 con 0x7efdb8005230 2024-03-08T06:04:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.176+0000 7efdcd600640 1 -- 172.21.15.40:0/151017267 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efdc400ac30 con 0x7efdb8005230 2024-03-08T06:04:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.177+0000 7efdcd600640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efdb006b500 0x7efdb006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.177+0000 7efdcd600640 1 -- 172.21.15.40:0/151017267 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efdc408a220 con 0x7efdb8005230 2024-03-08T06:04:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.177+0000 7efdd5000640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efdb006b500 0x7efdb006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.178+0000 7efdd5000640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efdb006b500 0x7efdb006d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efdd01f7260 tx=0x7efdbc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.181+0000 7efdcd600640 1 -- 172.21.15.40:0/151017267 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efdd00ad0e0 con 0x7efdb8005230 2024-03-08T06:04:50.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.381+0000 7efdd6e00640 1 -- 172.21.15.40:0/151017267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efdd0075090 con 0x7efdb8005230 2024-03-08T06:04:50.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.382+0000 7efdcd600640 1 -- 172.21.15.40:0/151017267 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7efdc4059020 con 0x7efdb8005230 2024-03-08T06:04:50.383 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:50.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 -- 172.21.15.40:0/151017267 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efdb006b500 msgr2=0x7efdb006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efdb006b500 0x7efdb006d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efdd01f7260 tx=0x7efdbc008040 comp rx=0 tx=0).stop 2024-03-08T06:04:50.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 -- 172.21.15.40:0/151017267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 msgr2=0x7efdd015adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 0x7efdd015adb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efdc400c790 tx=0x7efdc400cc50 comp rx=0 tx=0).stop 2024-03-08T06:04:50.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 -- 172.21.15.40:0/151017267 shutdown_connections 2024-03-08T06:04:50.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efdb006b500 0x7efdb006d9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdd00b36c0 0x7efdd00b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efdd01e52e0 0x7efdd01e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 --2- 172.21.15.40:0/151017267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdb8005230 0x7efdd015adb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 -- 172.21.15.40:0/151017267 >> 172.21.15.40:0/151017267 conn(0x7efdd00b1850 msgr2=0x7efdd00af540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 -- 172.21.15.40:0/151017267 shutdown_connections 2024-03-08T06:04:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.385+0000 7efdd6e00640 1 -- 172.21.15.40:0/151017267 wait complete. 2024-03-08T06:04:50.407 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.634+0000 7fe101800640 1 Processor -- start 2024-03-08T06:04:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.635+0000 7fe101800640 1 -- start start 2024-03-08T06:04:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.636+0000 7fe101800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc15de60 0x7fe0fc161930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.636+0000 7fe101800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe0fc0b9630 0x7fe0fc0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.636+0000 7fe101800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0fc0cee60 0x7fe0fc0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.636+0000 7fe101800640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc15e240 con 0x7fe0fc0cee60 2024-03-08T06:04:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.636+0000 7fe101800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc162030 con 0x7fe0fc0b9630 2024-03-08T06:04:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.636+0000 7fe101800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc162310 con 0x7fe0fc15de60 2024-03-08T06:04:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.636+0000 7fe100e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0fc0cee60 0x7fe0fc0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.636+0000 7fe100e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe0fc0cee60 0x7fe0fc0b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44122/0 (socket says 172.21.15.40:44122) 2024-03-08T06:04:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.636+0000 7fe100e00640 1 -- 172.21.15.40:0/2650601815 learned_addr learned my addr 172.21.15.40:0/2650601815 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.637+0000 7fe0fb400640 1 --2- 172.21.15.40:0/2650601815 >> v2:172.21.15.53:3300/0 conn(0x7fe0fc0b9630 0x7fe0fc0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.637+0000 7fe0fbe00640 1 --2- 172.21.15.40:0/2650601815 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc15de60 0x7fe0fc161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.637+0000 7fe100e00640 1 -- 172.21.15.40:0/2650601815 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc15de60 msgr2=0x7fe0fc161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.637+0000 7fe100e00640 1 --2- 172.21.15.40:0/2650601815 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc15de60 0x7fe0fc161930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.637+0000 7fe100e00640 1 -- 172.21.15.40:0/2650601815 >> v2:172.21.15.53:3300/0 conn(0x7fe0fc0b9630 msgr2=0x7fe0fc0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.637+0000 7fe100e00640 1 --2- 172.21.15.40:0/2650601815 >> v2:172.21.15.53:3300/0 conn(0x7fe0fc0b9630 0x7fe0fc0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.637+0000 7fe100e00640 1 -- 172.21.15.40:0/2650601815 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc1625f0 con 0x7fe0fc0cee60 2024-03-08T06:04:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.637+0000 7fe0fbe00640 1 --2- 172.21.15.40:0/2650601815 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc15de60 0x7fe0fc161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.638+0000 7fe100e00640 1 --2- 172.21.15.40:0/2650601815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0cee60 0x7fe0fc0b55d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0b6650 tx=0x7fe0f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be73debbd87b66e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.638+0000 7fe0faa00640 1 -- 172.21.15.40:0/2650601815 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4035020 con 0x7fe0fc0cee60 2024-03-08T06:04:50.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.638+0000 7fe0faa00640 1 -- 172.21.15.40:0/2650601815 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f4031870 con 0x7fe0fc0cee60 2024-03-08T06:04:50.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.638+0000 7fe0faa00640 1 -- 172.21.15.40:0/2650601815 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4031c40 con 0x7fe0fc0cee60 2024-03-08T06:04:50.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.639+0000 7fe101800640 1 -- 172.21.15.40:0/2650601815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0cee60 msgr2=0x7fe0fc0b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.639+0000 7fe101800640 1 --2- 172.21.15.40:0/2650601815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0cee60 0x7fe0fc0b55d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0b6650 tx=0x7fe0f402b910 comp rx=0 tx=0).stop 2024-03-08T06:04:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.639+0000 7fe101800640 1 -- 172.21.15.40:0/2650601815 shutdown_connections 2024-03-08T06:04:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.639+0000 7fe101800640 1 --2- 172.21.15.40:0/2650601815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0cee60 0x7fe0fc0b55d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.639+0000 7fe101800640 1 --2- 172.21.15.40:0/2650601815 >> v2:172.21.15.53:3300/0 conn(0x7fe0fc0b9630 0x7fe0fc0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.639+0000 7fe101800640 1 --2- 172.21.15.40:0/2650601815 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc15de60 0x7fe0fc161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.639+0000 7fe101800640 1 -- 172.21.15.40:0/2650601815 >> 172.21.15.40:0/2650601815 conn(0x7fe0fc06c500 msgr2=0x7fe0fc06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.639+0000 7fe101800640 1 -- 172.21.15.40:0/2650601815 shutdown_connections 2024-03-08T06:04:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.639+0000 7fe101800640 1 -- 172.21.15.40:0/2650601815 wait complete. 2024-03-08T06:04:50.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.641+0000 7fe101800640 1 Processor -- start 2024-03-08T06:04:50.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.642+0000 7fe101800640 1 -- start start 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.643+0000 7fe101800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc0b9630 0x7fe0fc0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.643+0000 7fe101800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0cee60 0x7fe0fc15ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.643+0000 7fe101800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0fc15de60 0x7fe0fc1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.643+0000 7fe101800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc0b5b10 con 0x7fe0fc0cee60 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.643+0000 7fe101800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc0b9a90 con 0x7fe0fc15de60 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.643+0000 7fe101800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc0b5c90 con 0x7fe0fc0b9630 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.643+0000 7fe0fb400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0cee60 0x7fe0fc15ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.643+0000 7fe0fb400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0cee60 0x7fe0fc15ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44138/0 (socket says 172.21.15.40:44138) 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.643+0000 7fe0fb400640 1 -- 172.21.15.40:0/1981277458 learned_addr learned my addr 172.21.15.40:0/1981277458 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.643+0000 7fe0fb400640 1 -- 172.21.15.40:0/1981277458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc0b9630 msgr2=0x7fe0fc0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.643+0000 7fe0fbe00640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc0b9630 0x7fe0fc0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe0fb400640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc0b9630 0x7fe0fc0cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe100e00640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0fc15de60 0x7fe0fc1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe0fb400640 1 -- 172.21.15.40:0/1981277458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0fc15de60 msgr2=0x7fe0fc1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe0fb400640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0fc15de60 0x7fe0fc1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe0fb400640 1 -- 172.21.15.40:0/1981277458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc0ccd60 con 0x7fe0fc0cee60 2024-03-08T06:04:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe0fbe00640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc0b9630 0x7fe0fc0cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe100e00640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0fc15de60 0x7fe0fc1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe0fb400640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0cee60 0x7fe0fc15ffa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4007e90 tx=0x7fe0e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe0f3e00640 1 -- 172.21.15.40:0/1981277458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4014620 con 0x7fe0fc0cee60 2024-03-08T06:04:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe0f3e00640 1 -- 172.21.15.40:0/1981277458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e4019070 con 0x7fe0fc0cee60 2024-03-08T06:04:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe0f3e00640 1 -- 172.21.15.40:0/1981277458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4014a00 con 0x7fe0fc0cee60 2024-03-08T06:04:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe101800640 1 -- 172.21.15.40:0/1981277458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc0cd040 con 0x7fe0fc0cee60 2024-03-08T06:04:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.644+0000 7fe101800640 1 -- 172.21.15.40:0/1981277458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc157710 con 0x7fe0fc0cee60 2024-03-08T06:04:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.646+0000 7fe101800640 1 -- 172.21.15.40:0/1981277458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc075090 con 0x7fe0fc0cee60 2024-03-08T06:04:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.647+0000 7fe0f3e00640 1 -- 172.21.15.40:0/1981277458 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0e4014ba0 con 0x7fe0fc0cee60 2024-03-08T06:04:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.648+0000 7fe0f3e00640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe0d806b4d0 0x7fe0d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.648+0000 7fe0f3e00640 1 -- 172.21.15.40:0/1981277458 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe0e408ab50 con 0x7fe0fc0cee60 2024-03-08T06:04:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.648+0000 7fe0fbe00640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe0d806b4d0 0x7fe0d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.649+0000 7fe0fbe00640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe0d806b4d0 0x7fe0d806d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec006fd0 tx=0x7fe0ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:50.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.652+0000 7fe0f3e00640 1 -- 172.21.15.40:0/1981277458 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0e4059940 con 0x7fe0fc0cee60 2024-03-08T06:04:50.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.854+0000 7fe101800640 1 -- 172.21.15.40:0/1981277458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0fc0c7420 con 0x7fe0fc0cee60 2024-03-08T06:04:50.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.855+0000 7fe0f3e00640 1 -- 172.21.15.40:0/1981277458 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0e4059240 con 0x7fe0fc0cee60 2024-03-08T06:04:50.856 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.858+0000 7fe101800640 1 -- 172.21.15.40:0/1981277458 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe0d806b4d0 msgr2=0x7fe0d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.858+0000 7fe101800640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe0d806b4d0 0x7fe0d806d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec006fd0 tx=0x7fe0ec008040 comp rx=0 tx=0).stop 2024-03-08T06:04:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.858+0000 7fe101800640 1 -- 172.21.15.40:0/1981277458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0cee60 msgr2=0x7fe0fc15ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.858+0000 7fe101800640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0cee60 0x7fe0fc15ffa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4007e90 tx=0x7fe0e400e4b0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.858+0000 7fe101800640 1 -- 172.21.15.40:0/1981277458 shutdown_connections 2024-03-08T06:04:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.858+0000 7fe101800640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe0d806b4d0 0x7fe0d806d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.858+0000 7fe101800640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0fc15de60 0x7fe0fc1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.858+0000 7fe101800640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0cee60 0x7fe0fc15ffa0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.858+0000 7fe101800640 1 --2- 172.21.15.40:0/1981277458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0fc0b9630 0x7fe0fc0cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.858+0000 7fe101800640 1 -- 172.21.15.40:0/1981277458 >> 172.21.15.40:0/1981277458 conn(0x7fe0fc06c500 msgr2=0x7fe0fc161850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.858+0000 7fe101800640 1 -- 172.21.15.40:0/1981277458 shutdown_connections 2024-03-08T06:04:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:50.859+0000 7fe101800640 1 -- 172.21.15.40:0/1981277458 wait complete. 2024-03-08T06:04:50.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:04:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.088+0000 7ff788600640 1 Processor -- start 2024-03-08T06:04:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.088+0000 7ff788600640 1 -- start start 2024-03-08T06:04:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.089+0000 7ff788600640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff78015df80 0x7ff7800ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.089+0000 7ff788600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff7800d10e0 0x7ff7800d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:51.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.089+0000 7ff788600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7800ce9a0 con 0x7ff7800b6e30 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.089+0000 7ff788600640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7800d2000 con 0x7ff78015df80 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.089+0000 7ff788600640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7800cec40 con 0x7ff7800d10e0 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.090+0000 7ff785c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff78015df80 0x7ff7800ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.090+0000 7ff785200640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff7800b6e30 0x7ff7800d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40154/0 (socket says 172.21.15.40:40154) 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.090+0000 7ff785c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff78015df80 0x7ff7800ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44150/0 (socket says 172.21.15.40:44150) 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.090+0000 7ff785200640 1 -- 172.21.15.40:0/307310224 learned_addr learned my addr 172.21.15.40:0/307310224 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.090+0000 7ff786600640 1 --2- 172.21.15.40:0/307310224 >> v2:172.21.15.53:3300/0 conn(0x7ff7800d10e0 0x7ff7800d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.090+0000 7ff785c00640 1 -- 172.21.15.40:0/307310224 >> v2:172.21.15.53:3300/0 conn(0x7ff7800d10e0 msgr2=0x7ff7800d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.090+0000 7ff785c00640 1 --2- 172.21.15.40:0/307310224 >> v2:172.21.15.53:3300/0 conn(0x7ff7800d10e0 0x7ff7800d2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.090+0000 7ff785c00640 1 -- 172.21.15.40:0/307310224 >> v1:172.21.15.40:6789/0 conn(0x7ff7800b6e30 legacy=0x7ff7800d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.090+0000 7ff785c00640 1 -- 172.21.15.40:0/307310224 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800cef20 con 0x7ff78015df80 2024-03-08T06:04:51.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.090+0000 7ff77fe00640 1 -- 172.21.15.40:0/307310224 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801739594 0 0) 0x7ff7800ce9a0 con 0x7ff7800b6e30 2024-03-08T06:04:51.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.091+0000 7ff785c00640 1 --2- 172.21.15.40:0/307310224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff78015df80 0x7ff7800ce460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff7800d2180 tx=0x7ff77402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4207c7f3124b96 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:51.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.092+0000 7ff77fe00640 1 -- 172.21.15.40:0/307310224 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7740315a0 con 0x7ff78015df80 2024-03-08T06:04:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.092+0000 7ff77fe00640 1 -- 172.21.15.40:0/307310224 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff774037070 con 0x7ff78015df80 2024-03-08T06:04:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.092+0000 7ff77fe00640 1 -- 172.21.15.40:0/307310224 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774031960 con 0x7ff78015df80 2024-03-08T06:04:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.092+0000 7ff788600640 1 -- 172.21.15.40:0/307310224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff78015df80 msgr2=0x7ff7800ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.092+0000 7ff788600640 1 --2- 172.21.15.40:0/307310224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff78015df80 0x7ff7800ce460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff7800d2180 tx=0x7ff77402b840 comp rx=0 tx=0).stop 2024-03-08T06:04:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.092+0000 7ff788600640 1 -- 172.21.15.40:0/307310224 shutdown_connections 2024-03-08T06:04:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.092+0000 7ff788600640 1 --2- 172.21.15.40:0/307310224 >> v2:172.21.15.53:3300/0 conn(0x7ff7800d10e0 0x7ff7800d2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.092+0000 7ff788600640 1 --2- 172.21.15.40:0/307310224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff78015df80 0x7ff7800ce460 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.092+0000 7ff788600640 1 -- 172.21.15.40:0/307310224 >> 172.21.15.40:0/307310224 conn(0x7ff78006c5e0 msgr2=0x7ff78006c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.093+0000 7ff788600640 1 -- 172.21.15.40:0/307310224 shutdown_connections 2024-03-08T06:04:51.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.093+0000 7ff788600640 1 -- 172.21.15.40:0/307310224 wait complete. 2024-03-08T06:04:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.094+0000 7ff788600640 1 Processor -- start 2024-03-08T06:04:51.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.095+0000 7ff788600640 1 -- start start 2024-03-08T06:04:51.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.095+0000 7ff788600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800d10e0 0x7ff7800c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:51.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff788600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff78015df80 0x7ff7801f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:51.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff788600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7801f5280 0x7ff7800c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:51.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff788600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff780161ae0 con 0x7ff7801f5280 2024-03-08T06:04:51.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff788600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78015e360 con 0x7ff7800d10e0 2024-03-08T06:04:51.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff788600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff780161c60 con 0x7ff78015df80 2024-03-08T06:04:51.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff786600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7801f5280 0x7ff7800c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:51.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff786600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7801f5280 0x7ff7800c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44162/0 (socket says 172.21.15.40:44162) 2024-03-08T06:04:51.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff786600640 1 -- 172.21.15.40:0/613570231 learned_addr learned my addr 172.21.15.40:0/613570231 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:51.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff785c00640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800d10e0 0x7ff7800c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff785200640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff78015df80 0x7ff7801f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff786600640 1 -- 172.21.15.40:0/613570231 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff78015df80 msgr2=0x7ff7801f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff786600640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff78015df80 0x7ff7801f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff786600640 1 -- 172.21.15.40:0/613570231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800d10e0 msgr2=0x7ff7800c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff786600640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800d10e0 0x7ff7800c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.096+0000 7ff786600640 1 -- 172.21.15.40:0/613570231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800c74a0 con 0x7ff7801f5280 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.097+0000 7ff785200640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff78015df80 0x7ff7801f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.097+0000 7ff786600640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7801f5280 0x7ff7800c8260 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff77800cd70 tx=0x7ff77800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.097+0000 7ff77d600640 1 -- 172.21.15.40:0/613570231 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778003b90 con 0x7ff7801f5280 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.097+0000 7ff77d600640 1 -- 172.21.15.40:0/613570231 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff778003d30 con 0x7ff7801f5280 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.097+0000 7ff77d600640 1 -- 172.21.15.40:0/613570231 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778004240 con 0x7ff7801f5280 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.097+0000 7ff788600640 1 -- 172.21.15.40:0/613570231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7800c7780 con 0x7ff7801f5280 2024-03-08T06:04:51.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.097+0000 7ff788600640 1 -- 172.21.15.40:0/613570231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7800c9ee0 con 0x7ff7801f5280 2024-03-08T06:04:51.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.099+0000 7ff788600640 1 -- 172.21.15.40:0/613570231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff780067cb0 con 0x7ff7801f5280 2024-03-08T06:04:51.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.100+0000 7ff77d600640 1 -- 172.21.15.40:0/613570231 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7780043e0 con 0x7ff7801f5280 2024-03-08T06:04:51.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.101+0000 7ff77d600640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff76006b4d0 0x7ff76006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:51.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.101+0000 7ff77d600640 1 -- 172.21.15.40:0/613570231 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff77808ae10 con 0x7ff7801f5280 2024-03-08T06:04:51.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.101+0000 7ff785c00640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff76006b4d0 0x7ff76006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:51.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.102+0000 7ff785c00640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff76006b4d0 0x7ff76006d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff774036000 tx=0x7ff774008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:51.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.106+0000 7ff77d600640 1 -- 172.21.15.40:0/613570231 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff77808e030 con 0x7ff7801f5280 2024-03-08T06:04:51.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.318+0000 7ff788600640 1 -- 172.21.15.40:0/613570231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff7800b85e0 con 0x7ff7801f5280 2024-03-08T06:04:51.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.319+0000 7ff77d600640 1 -- 172.21.15.40:0/613570231 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff77801d340 con 0x7ff7801f5280 2024-03-08T06:04:51.320 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-03-08T06:04:51.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.321+0000 7ff788600640 1 -- 172.21.15.40:0/613570231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff76006b4d0 msgr2=0x7ff76006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:51.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.321+0000 7ff788600640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff76006b4d0 0x7ff76006d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff774036000 tx=0x7ff774008030 comp rx=0 tx=0).stop 2024-03-08T06:04:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.321+0000 7ff788600640 1 -- 172.21.15.40:0/613570231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7801f5280 msgr2=0x7ff7800c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.321+0000 7ff788600640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7801f5280 0x7ff7800c8260 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff77800cd70 tx=0x7ff77800a660 comp rx=0 tx=0).stop 2024-03-08T06:04:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.321+0000 7ff788600640 1 -- 172.21.15.40:0/613570231 shutdown_connections 2024-03-08T06:04:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.321+0000 7ff788600640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff76006b4d0 0x7ff76006d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.321+0000 7ff788600640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7801f5280 0x7ff7800c8260 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.321+0000 7ff788600640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff78015df80 0x7ff7801f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.321+0000 7ff788600640 1 --2- 172.21.15.40:0/613570231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800d10e0 0x7ff7800c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.321+0000 7ff788600640 1 -- 172.21.15.40:0/613570231 >> 172.21.15.40:0/613570231 conn(0x7ff78006c5e0 msgr2=0x7ff7800b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.321+0000 7ff788600640 1 -- 172.21.15.40:0/613570231 shutdown_connections 2024-03-08T06:04:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.322+0000 7ff788600640 1 -- 172.21.15.40:0/613570231 wait complete. 2024-03-08T06:04:51.341 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-08T06:04:51.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:51.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-08T06:04:51.442 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:04:51.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:51.443 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-08T06:04:51.494 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:04:51.494 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:51.553 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:04:51.553 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:51.601 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:04:51.602 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:04:51.702 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.934+0000 7fe35b200640 1 Processor -- start 2024-03-08T06:04:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.934+0000 7fe35b200640 1 -- start start 2024-03-08T06:04:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.935+0000 7fe35b200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe35c15ed80 0x7fe35c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.935+0000 7fe35b200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c06ec90 con 0x7fe35c06e870 2024-03-08T06:04:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.935+0000 7fe35b200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c0cd020 con 0x7fe35c15a770 2024-03-08T06:04:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.935+0000 7fe35b200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35c0ccdd0 con 0x7fe35c15ed80 2024-03-08T06:04:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.936+0000 7fe351400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe35c15ed80 0x7fe35c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.936+0000 7fe35a800640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe35c06e870 0x7fe35c15ab50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40166/0 (socket says 172.21.15.40:40166) 2024-03-08T06:04:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.936+0000 7fe35a800640 1 -- 172.21.15.40:0/2587492471 learned_addr learned my addr 172.21.15.40:0/2587492471 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.937+0000 7fe351400640 1 -- 172.21.15.40:0/2587492471 >> v1:172.21.15.53:6789/0 conn(0x7fe35c15a770 legacy=0x7fe35c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.937+0000 7fe351400640 1 -- 172.21.15.40:0/2587492471 >> v1:172.21.15.40:6789/0 conn(0x7fe35c06e870 legacy=0x7fe35c15ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.937+0000 7fe351400640 1 -- 172.21.15.40:0/2587492471 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0cd300 con 0x7fe35c15ed80 2024-03-08T06:04:51.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.938+0000 7fe351400640 1 --2- 172.21.15.40:0/2587492471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35c15ed80 0x7fe35c06e330 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3440097a0 tx=0x7fe34402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f95f12a3bc292f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:51.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.938+0000 7fe359400640 1 -- 172.21.15.40:0/2587492471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe344034070 con 0x7fe35c15ed80 2024-03-08T06:04:51.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.938+0000 7fe359400640 1 -- 172.21.15.40:0/2587492471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe344030610 con 0x7fe35c15ed80 2024-03-08T06:04:51.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.938+0000 7fe359400640 1 -- 172.21.15.40:0/2587492471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3440309e0 con 0x7fe35c15ed80 2024-03-08T06:04:51.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.939+0000 7fe35b200640 1 -- 172.21.15.40:0/2587492471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35c15ed80 msgr2=0x7fe35c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:51.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.939+0000 7fe35b200640 1 --2- 172.21.15.40:0/2587492471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35c15ed80 0x7fe35c06e330 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3440097a0 tx=0x7fe34402b960 comp rx=0 tx=0).stop 2024-03-08T06:04:51.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.939+0000 7fe35b200640 1 -- 172.21.15.40:0/2587492471 shutdown_connections 2024-03-08T06:04:51.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.939+0000 7fe35b200640 1 --2- 172.21.15.40:0/2587492471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35c15ed80 0x7fe35c06e330 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.939+0000 7fe35b200640 1 -- 172.21.15.40:0/2587492471 >> 172.21.15.40:0/2587492471 conn(0x7fe35c0b1850 msgr2=0x7fe35c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:51.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.939+0000 7fe35b200640 1 -- 172.21.15.40:0/2587492471 shutdown_connections 2024-03-08T06:04:51.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.939+0000 7fe35b200640 1 -- 172.21.15.40:0/2587492471 wait complete. 2024-03-08T06:04:51.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.940+0000 7fe35b200640 1 Processor -- start 2024-03-08T06:04:51.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.941+0000 7fe35b200640 1 -- start start 2024-03-08T06:04:51.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.941+0000 7fe35b200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe35c06e870 0x7fe35c14d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe35c15a770 0x7fe35c144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35b200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35c15ed80 0x7fe35c0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35b200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c15f200 con 0x7fe35c15ed80 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c15e8f0 con 0x7fe35c15a770 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35b200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0b4e00 con 0x7fe35c06e870 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35a800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35c15ed80 0x7fe35c0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35a800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35c15ed80 0x7fe35c0adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44184/0 (socket says 172.21.15.40:44184) 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35a800640 1 -- 172.21.15.40:0/3636861780 learned_addr learned my addr 172.21.15.40:0/3636861780 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe351400640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe35c15a770 0x7fe35c144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35a800640 1 -- 172.21.15.40:0/3636861780 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe35c06e870 msgr2=0x7fe35c14d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35a800640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe35c06e870 0x7fe35c14d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35a800640 1 -- 172.21.15.40:0/3636861780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe35c15a770 msgr2=0x7fe35c144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35a800640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe35c15a770 0x7fe35c144020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.942+0000 7fe35a800640 1 -- 172.21.15.40:0/3636861780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c1449b0 con 0x7fe35c15ed80 2024-03-08T06:04:51.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.943+0000 7fe35a800640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35c15ed80 0x7fe35c0adb80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe35400ca80 tx=0x7fe35400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.943+0000 7fe352a00640 1 -- 172.21.15.40:0/3636861780 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354017070 con 0x7fe35c15ed80 2024-03-08T06:04:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.943+0000 7fe352a00640 1 -- 172.21.15.40:0/3636861780 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35400ac30 con 0x7fe35c15ed80 2024-03-08T06:04:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.943+0000 7fe352a00640 1 -- 172.21.15.40:0/3636861780 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354012600 con 0x7fe35c15ed80 2024-03-08T06:04:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.943+0000 7fe35b200640 1 -- 172.21.15.40:0/3636861780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c144c90 con 0x7fe35c15ed80 2024-03-08T06:04:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.943+0000 7fe35b200640 1 -- 172.21.15.40:0/3636861780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c145250 con 0x7fe35c15ed80 2024-03-08T06:04:51.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.944+0000 7fe35b200640 1 -- 172.21.15.40:0/3636861780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c075100 con 0x7fe35c15ed80 2024-03-08T06:04:51.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.945+0000 7fe352a00640 1 -- 172.21.15.40:0/3636861780 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe35401a070 con 0x7fe35c15ed80 2024-03-08T06:04:51.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.946+0000 7fe352a00640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe33806b520 0x7fe33806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:51.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.946+0000 7fe352a00640 1 -- 172.21.15.40:0/3636861780 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe354089ea0 con 0x7fe35c15ed80 2024-03-08T06:04:51.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.946+0000 7fe359e00640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe33806b520 0x7fe33806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:51.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.947+0000 7fe359e00640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe33806b520 0x7fe33806d9e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe34c000f60 tx=0x7fe34c0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:51.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:51.951+0000 7fe352a00640 1 -- 172.21.15.40:0/3636861780 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe35400ae20 con 0x7fe35c15ed80 2024-03-08T06:04:52.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.153+0000 7fe35b200640 1 -- 172.21.15.40:0/3636861780 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe35c0b3a10 con 0x7fe35c15ed80 2024-03-08T06:04:52.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.156+0000 7fe352a00640 1 -- 172.21.15.40:0/3636861780 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16770 (secure 0 0 0) 0x7fe3540586c0 con 0x7fe35c15ed80 2024-03-08T06:04:52.157 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:52.157 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:48.785806+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:52.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.158+0000 7fe35b200640 1 -- 172.21.15.40:0/3636861780 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe33806b520 msgr2=0x7fe33806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.158+0000 7fe35b200640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe33806b520 0x7fe33806d9e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe34c000f60 tx=0x7fe34c0079b0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.158+0000 7fe35b200640 1 -- 172.21.15.40:0/3636861780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35c15ed80 msgr2=0x7fe35c0adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.158+0000 7fe35b200640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35c15ed80 0x7fe35c0adb80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe35400ca80 tx=0x7fe35400cf40 comp rx=0 tx=0).stop 2024-03-08T06:04:52.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.159+0000 7fe35b200640 1 -- 172.21.15.40:0/3636861780 shutdown_connections 2024-03-08T06:04:52.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.159+0000 7fe35b200640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe33806b520 0x7fe33806d9e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.159+0000 7fe35b200640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35c15ed80 0x7fe35c0adb80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.159+0000 7fe35b200640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe35c15a770 0x7fe35c144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.159+0000 7fe35b200640 1 --2- 172.21.15.40:0/3636861780 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe35c06e870 0x7fe35c14d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.159+0000 7fe35b200640 1 -- 172.21.15.40:0/3636861780 >> 172.21.15.40:0/3636861780 conn(0x7fe35c0b1850 msgr2=0x7fe35c06d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:52.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.159+0000 7fe35b200640 1 -- 172.21.15.40:0/3636861780 shutdown_connections 2024-03-08T06:04:52.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.159+0000 7fe35b200640 1 -- 172.21.15.40:0/3636861780 wait complete. 2024-03-08T06:04:52.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:52.181 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.406+0000 7f40d1600640 1 Processor -- start 2024-03-08T06:04:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.407+0000 7f40d1600640 1 -- start start 2024-03-08T06:04:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.408+0000 7f40d1600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f40cc071950 0x7f40cc071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.408+0000 7f40d1600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40cc0cd5a0 con 0x7f40cc1455b0 2024-03-08T06:04:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.408+0000 7f40d1600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40cc0ccfe0 con 0x7f40cc1604f0 2024-03-08T06:04:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.408+0000 7f40d1600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0ccda0 con 0x7f40cc071950 2024-03-08T06:04:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.409+0000 7f40caa00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f40cc1455b0 0x7f40cc15c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40170/0 (socket says 172.21.15.40:40170) 2024-03-08T06:04:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.409+0000 7f40caa00640 1 -- 172.21.15.40:0/1256259677 learned_addr learned my addr 172.21.15.40:0/1256259677 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.409+0000 7f40cb400640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f40cc071950 0x7f40cc071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.409+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207265593 0 0) 0x7f40cc0cd5a0 con 0x7f40cc1455b0 2024-03-08T06:04:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.409+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40ac003520 con 0x7f40cc1455b0 2024-03-08T06:04:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.409+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669926473 0 0) 0x7f40ac003520 con 0x7f40cc1455b0 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.409+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f40cc071950 msgr2=0x7f40cc071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40c9600640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f40cc071950 0x7f40cc071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 >> v1:172.21.15.53:6789/0 conn(0x7f40cc1604f0 legacy=0x7f40cc0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc0cd2c0 con 0x7f40cc1455b0 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f40bc001670 con 0x7f40cc1455b0 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 >> v1:172.21.15.40:6789/0 conn(0x7f40cc1455b0 legacy=0x7f40cc15c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40c9600640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40ac0039e0 0x7f40ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40c9600640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40ac004800 0x7f40ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40c9600640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f40cc071950 0x7f40ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0ccda0 con 0x7f40ac0039e0 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac00baa0 con 0x7f40ac004800 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac00bca0 con 0x7f40cc071950 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40caa00640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40ac0039e0 0x7f40ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40ca000640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40ac004800 0x7f40ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40caa00640 1 -- 172.21.15.40:0/1256259677 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f40cc071950 msgr2=0x7f40ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:52.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40caa00640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f40cc071950 0x7f40ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40caa00640 1 -- 172.21.15.40:0/1256259677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40ac004800 msgr2=0x7f40ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:52.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40caa00640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40ac004800 0x7f40ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.410+0000 7f40caa00640 1 -- 172.21.15.40:0/1256259677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac003520 con 0x7f40ac0039e0 2024-03-08T06:04:52.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.411+0000 7f40ca000640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40ac004800 0x7f40ac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:52.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.412+0000 7f40caa00640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40ac0039e0 0x7f40ac003e00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00c210 tx=0x7f40ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46205ee9d5ee3a86 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:52.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.412+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc007cf0 con 0x7f40ac0039e0 2024-03-08T06:04:52.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.412+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc030070 con 0x7f40ac0039e0 2024-03-08T06:04:52.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.412+0000 7f40c9600640 1 -- 172.21.15.40:0/1256259677 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc005db0 con 0x7f40ac0039e0 2024-03-08T06:04:52.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.413+0000 7f40d1600640 1 -- 172.21.15.40:0/1256259677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40ac0039e0 msgr2=0x7f40ac003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.413+0000 7f40d1600640 1 --2- 172.21.15.40:0/1256259677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40ac0039e0 0x7f40ac003e00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00c210 tx=0x7f40ac005830 comp rx=0 tx=0).stop 2024-03-08T06:04:52.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.413+0000 7f40cb400640 1 -- 172.21.15.40:0/1256259677 reap_dead start 2024-03-08T06:04:52.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.413+0000 7f40d1600640 1 -- 172.21.15.40:0/1256259677 shutdown_connections 2024-03-08T06:04:52.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.413+0000 7f40d1600640 1 -- 172.21.15.40:0/1256259677 >> 172.21.15.40:0/1256259677 conn(0x7f40cc06ae30 msgr2=0x7f40cc06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:52.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.413+0000 7f40d1600640 1 -- 172.21.15.40:0/1256259677 shutdown_connections 2024-03-08T06:04:52.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.413+0000 7f40d1600640 1 -- 172.21.15.40:0/1256259677 wait complete. 2024-03-08T06:04:52.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.415+0000 7f40d1600640 1 Processor -- start 2024-03-08T06:04:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.415+0000 7f40d1600640 1 -- start start 2024-03-08T06:04:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40d1600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f40ac0039e0 0x7f40cc15b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40d1600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc0b1940 0x7f40cc0b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40d1600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40cc0cb340 0x7f40cc0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40d1600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc145990 con 0x7f40cc0b1940 2024-03-08T06:04:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40d1600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0cdc20 con 0x7f40cc0cb340 2024-03-08T06:04:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40d1600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc071d90 con 0x7f40ac0039e0 2024-03-08T06:04:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40ca000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc0b1940 0x7f40cc0b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40ca000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc0b1940 0x7f40cc0b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44198/0 (socket says 172.21.15.40:44198) 2024-03-08T06:04:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40ca000640 1 -- 172.21.15.40:0/2624626099 learned_addr learned my addr 172.21.15.40:0/2624626099 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40cb400640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40cc0cb340 0x7f40cc0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40ca000640 1 -- 172.21.15.40:0/2624626099 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f40ac0039e0 msgr2=0x7f40cc15b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40ca000640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f40ac0039e0 0x7f40cc15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40ca000640 1 -- 172.21.15.40:0/2624626099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40cc0cb340 msgr2=0x7f40cc0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.416+0000 7f40ca000640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40cc0cb340 0x7f40cc0cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.417+0000 7f40ca000640 1 -- 172.21.15.40:0/2624626099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc0cc000 con 0x7f40cc0b1940 2024-03-08T06:04:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.417+0000 7f40ca000640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc0b1940 0x7f40cc0b1da0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f40b400db70 tx=0x7f40b400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.417+0000 7f40c2a00640 1 -- 172.21.15.40:0/2624626099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b400de70 con 0x7f40cc0b1940 2024-03-08T06:04:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.417+0000 7f40c2a00640 1 -- 172.21.15.40:0/2624626099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b400bbb0 con 0x7f40cc0b1940 2024-03-08T06:04:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.417+0000 7f40c2a00640 1 -- 172.21.15.40:0/2624626099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4006b50 con 0x7f40cc0b1940 2024-03-08T06:04:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.417+0000 7f40d1600640 1 -- 172.21.15.40:0/2624626099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40cc0cc2e0 con 0x7f40cc0b1940 2024-03-08T06:04:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.417+0000 7f40d1600640 1 -- 172.21.15.40:0/2624626099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40cc0cc8a0 con 0x7f40cc0b1940 2024-03-08T06:04:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.419+0000 7f40d1600640 1 -- 172.21.15.40:0/2624626099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40cc0b22e0 con 0x7f40cc0b1940 2024-03-08T06:04:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.419+0000 7f40c2a00640 1 -- 172.21.15.40:0/2624626099 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40b401a020 con 0x7f40cc0b1940 2024-03-08T06:04:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.420+0000 7f40c2a00640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f40a806b4d0 0x7f40a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.420+0000 7f40c2a00640 1 -- 172.21.15.40:0/2624626099 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f40b4089ef0 con 0x7f40cc0b1940 2024-03-08T06:04:52.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.421+0000 7f40caa00640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f40a806b4d0 0x7f40a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.421+0000 7f40caa00640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f40a806b4d0 0x7f40a806d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f40bc034000 tx=0x7f40bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.425+0000 7f40c2a00640 1 -- 172.21.15.40:0/2624626099 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40b4058c50 con 0x7f40cc0b1940 2024-03-08T06:04:52.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.628+0000 7f40d1600640 1 -- 172.21.15.40:0/2624626099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40cc067cb0 con 0x7f40cc0b1940 2024-03-08T06:04:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.629+0000 7f40c2a00640 1 -- 172.21.15.40:0/2624626099 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40b4058550 con 0x7f40cc0b1940 2024-03-08T06:04:52.630 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.632+0000 7f40d1600640 1 -- 172.21.15.40:0/2624626099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f40a806b4d0 msgr2=0x7f40a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.632+0000 7f40d1600640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f40a806b4d0 0x7f40a806d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f40bc034000 tx=0x7f40bc02e000 comp rx=0 tx=0).stop 2024-03-08T06:04:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.632+0000 7f40d1600640 1 -- 172.21.15.40:0/2624626099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc0b1940 msgr2=0x7f40cc0b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.632+0000 7f40d1600640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc0b1940 0x7f40cc0b1da0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f40b400db70 tx=0x7f40b400b4a0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.632+0000 7f40d1600640 1 -- 172.21.15.40:0/2624626099 shutdown_connections 2024-03-08T06:04:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.632+0000 7f40d1600640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f40a806b4d0 0x7f40a806d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.632+0000 7f40d1600640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40cc0cb340 0x7f40cc0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.632+0000 7f40d1600640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc0b1940 0x7f40cc0b1da0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.632+0000 7f40d1600640 1 --2- 172.21.15.40:0/2624626099 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f40ac0039e0 0x7f40cc15b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.632+0000 7f40d1600640 1 -- 172.21.15.40:0/2624626099 >> 172.21.15.40:0/2624626099 conn(0x7f40cc06ae30 msgr2=0x7f40cc06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.633+0000 7f40d1600640 1 -- 172.21.15.40:0/2624626099 shutdown_connections 2024-03-08T06:04:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.633+0000 7f40d1600640 1 -- 172.21.15.40:0/2624626099 wait complete. 2024-03-08T06:04:52.653 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:52.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.873+0000 7fb7f5c00640 1 Processor -- start 2024-03-08T06:04:52.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.874+0000 7fb7f5c00640 1 -- start start 2024-03-08T06:04:52.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.875+0000 7fb7f5c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7f009aa90 0x7fb7f01632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.875+0000 7fb7f5c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f006bd60 0x7fb7f00cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.876+0000 7fb7f5c00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f0099cd0 con 0x7fb7f00c49c0 2024-03-08T06:04:52.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.876+0000 7fb7f5c00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f0099fb0 con 0x7fb7f009aa90 2024-03-08T06:04:52.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.876+0000 7fb7f5c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f009a290 con 0x7fb7f006bd60 2024-03-08T06:04:52.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.876+0000 7fb7ef400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7f009aa90 0x7fb7f01632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.876+0000 7fb7ef400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7f009aa90 0x7fb7f01632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44212/0 (socket says 172.21.15.40:44212) 2024-03-08T06:04:52.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.876+0000 7fb7ef400640 1 -- 172.21.15.40:0/1582808255 learned_addr learned my addr 172.21.15.40:0/1582808255 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:52.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.876+0000 7fb7ef400640 1 -- 172.21.15.40:0/1582808255 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f006bd60 msgr2=0x7fb7f00cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:52.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.876+0000 7fb7f5200640 1 --2- 172.21.15.40:0/1582808255 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f006bd60 0x7fb7f00cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.876+0000 7fb7ef400640 1 --2- 172.21.15.40:0/1582808255 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f006bd60 0x7fb7f00cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.877+0000 7fb7ef400640 1 -- 172.21.15.40:0/1582808255 >> v1:172.21.15.53:6789/0 conn(0x7fb7f00c49c0 legacy=0x7fb7f00c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:52.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.877+0000 7fb7ef400640 1 -- 172.21.15.40:0/1582808255 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f009a570 con 0x7fb7f009aa90 2024-03-08T06:04:52.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.877+0000 7fb7f5200640 1 --2- 172.21.15.40:0/1582808255 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f006bd60 0x7fb7f00cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:52.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.878+0000 7fb7ef400640 1 --2- 172.21.15.40:0/1582808255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f009aa90 0x7fb7f01632f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009870 tx=0x7fb7d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafcc5d62651649f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:52.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.878+0000 7fb7eea00640 1 -- 172.21.15.40:0/1582808255 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8035020 con 0x7fb7f009aa90 2024-03-08T06:04:52.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.878+0000 7fb7eea00640 1 -- 172.21.15.40:0/1582808255 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d8031640 con 0x7fb7f009aa90 2024-03-08T06:04:52.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.878+0000 7fb7eea00640 1 -- 172.21.15.40:0/1582808255 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d802d040 con 0x7fb7f009aa90 2024-03-08T06:04:52.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.879+0000 7fb7f5c00640 1 -- 172.21.15.40:0/1582808255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f009aa90 msgr2=0x7fb7f01632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.879+0000 7fb7f5c00640 1 --2- 172.21.15.40:0/1582808255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f009aa90 0x7fb7f01632f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009870 tx=0x7fb7d802b8f0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.879+0000 7fb7f5c00640 1 -- 172.21.15.40:0/1582808255 shutdown_connections 2024-03-08T06:04:52.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.879+0000 7fb7f5c00640 1 --2- 172.21.15.40:0/1582808255 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f006bd60 0x7fb7f00cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.879+0000 7fb7f5c00640 1 --2- 172.21.15.40:0/1582808255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f009aa90 0x7fb7f01632f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.879+0000 7fb7f5c00640 1 -- 172.21.15.40:0/1582808255 >> 172.21.15.40:0/1582808255 conn(0x7fb7f00b34e0 msgr2=0x7fb7f00b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:52.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.879+0000 7fb7f5c00640 1 -- 172.21.15.40:0/1582808255 shutdown_connections 2024-03-08T06:04:52.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.879+0000 7fb7f5c00640 1 -- 172.21.15.40:0/1582808255 wait complete. 2024-03-08T06:04:52.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.881+0000 7fb7f5c00640 1 Processor -- start 2024-03-08T06:04:52.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.881+0000 7fb7f5c00640 1 -- start start 2024-03-08T06:04:52.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.882+0000 7fb7f5c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7f006bd60 0x7fb7f0166530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.882+0000 7fb7f5c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f009aa90 0x7fb7f015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.882+0000 7fb7f5c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f00c49c0 0x7fb7f01ed200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.882+0000 7fb7f5c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f006c680 con 0x7fb7f00c49c0 2024-03-08T06:04:52.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.882+0000 7fb7f5c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f006c500 con 0x7fb7f006bd60 2024-03-08T06:04:52.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.882+0000 7fb7f5c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f015caf0 con 0x7fb7f009aa90 2024-03-08T06:04:52.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.882+0000 7fb7f5200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f00c49c0 0x7fb7f01ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.882+0000 7fb7f5200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f00c49c0 0x7fb7f01ed200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44218/0 (socket says 172.21.15.40:44218) 2024-03-08T06:04:52.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.882+0000 7fb7f5200640 1 -- 172.21.15.40:0/4103379450 learned_addr learned my addr 172.21.15.40:0/4103379450 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:52.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7efe00640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7f006bd60 0x7fb7f0166530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7ef400640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f009aa90 0x7fb7f015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7f5200640 1 -- 172.21.15.40:0/4103379450 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f009aa90 msgr2=0x7fb7f015b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7f5200640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f009aa90 0x7fb7f015b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7f5200640 1 -- 172.21.15.40:0/4103379450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7f006bd60 msgr2=0x7fb7f0166530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7f5200640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7f006bd60 0x7fb7f0166530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7f5200640 1 -- 172.21.15.40:0/4103379450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f00c4da0 con 0x7fb7f00c49c0 2024-03-08T06:04:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7ef400640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f009aa90 0x7fb7f015b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7f5200640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f00c49c0 0x7fb7f01ed200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800c820 tx=0x7fb7e800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7e7e00640 1 -- 172.21.15.40:0/4103379450 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8015070 con 0x7fb7f00c49c0 2024-03-08T06:04:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7e7e00640 1 -- 172.21.15.40:0/4103379450 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e8007a30 con 0x7fb7f00c49c0 2024-03-08T06:04:52.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7e7e00640 1 -- 172.21.15.40:0/4103379450 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8007e20 con 0x7fb7f00c49c0 2024-03-08T06:04:52.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.883+0000 7fb7f5c00640 1 -- 172.21.15.40:0/4103379450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f00c5080 con 0x7fb7f00c49c0 2024-03-08T06:04:52.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.884+0000 7fb7f5c00640 1 -- 172.21.15.40:0/4103379450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f00c5640 con 0x7fb7f00c49c0 2024-03-08T06:04:52.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.885+0000 7fb7f5c00640 1 -- 172.21.15.40:0/4103379450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f0075090 con 0x7fb7f00c49c0 2024-03-08T06:04:52.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.886+0000 7fb7e7e00640 1 -- 172.21.15.40:0/4103379450 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7e8018070 con 0x7fb7f00c49c0 2024-03-08T06:04:52.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.887+0000 7fb7e7e00640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7cc06b4d0 0x7fb7cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:52.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.887+0000 7fb7e7e00640 1 -- 172.21.15.40:0/4103379450 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb7e808a2b0 con 0x7fb7f00c49c0 2024-03-08T06:04:52.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.888+0000 7fb7efe00640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7cc06b4d0 0x7fb7cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:52.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.888+0000 7fb7efe00640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7cc06b4d0 0x7fb7cc06d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009f20 tx=0x7fb7e00094d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:52.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:52.892+0000 7fb7e7e00640 1 -- 172.21.15.40:0/4103379450 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7e8059010 con 0x7fb7f00c49c0 2024-03-08T06:04:53.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.091+0000 7fb7f5c00640 1 -- 172.21.15.40:0/4103379450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7f00b4360 con 0x7fb7f00c49c0 2024-03-08T06:04:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.092+0000 7fb7e7e00640 1 -- 172.21.15.40:0/4103379450 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7e8058910 con 0x7fb7f00c49c0 2024-03-08T06:04:53.093 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 -- 172.21.15.40:0/4103379450 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7cc06b4d0 msgr2=0x7fb7cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7cc06b4d0 0x7fb7cc06d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009f20 tx=0x7fb7e00094d0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 -- 172.21.15.40:0/4103379450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f00c49c0 msgr2=0x7fb7f01ed200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f00c49c0 0x7fb7f01ed200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800c820 tx=0x7fb7e800cce0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 -- 172.21.15.40:0/4103379450 shutdown_connections 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb7cc06b4d0 0x7fb7cc06d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f00c49c0 0x7fb7f01ed200 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb7f009aa90 0x7fb7f015b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 --2- 172.21.15.40:0/4103379450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7f006bd60 0x7fb7f0166530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 -- 172.21.15.40:0/4103379450 >> 172.21.15.40:0/4103379450 conn(0x7fb7f00b34e0 msgr2=0x7fb7f00af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 -- 172.21.15.40:0/4103379450 shutdown_connections 2024-03-08T06:04:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.095+0000 7fb7f5c00640 1 -- 172.21.15.40:0/4103379450 wait complete. 2024-03-08T06:04:53.115 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.341+0000 7f1cb0c00640 1 Processor -- start 2024-03-08T06:04:53.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.342+0000 7f1cb0c00640 1 -- start start 2024-03-08T06:04:53.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.343+0000 7f1cb0c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cac147040 0x7f1cac15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.343+0000 7f1cb0c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1cac0716e0 0x7f1cac071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.343+0000 7f1cb0c00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac15e840 con 0x7f1cac0b2970 2024-03-08T06:04:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.343+0000 7f1cb0c00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cac15eb20 con 0x7f1cac147040 2024-03-08T06:04:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.343+0000 7f1cb0c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac15ee00 con 0x7f1cac0716e0 2024-03-08T06:04:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.344+0000 7f1cab400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cac147040 0x7f1cac15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.344+0000 7f1cab400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cac147040 0x7f1cac15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44228/0 (socket says 172.21.15.40:44228) 2024-03-08T06:04:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.344+0000 7f1cab400640 1 -- 172.21.15.40:0/457233039 learned_addr learned my addr 172.21.15.40:0/457233039 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.344+0000 7f1cabe00640 1 --2- 172.21.15.40:0/457233039 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1cac0716e0 0x7f1cac071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.344+0000 7f1cab400640 1 -- 172.21.15.40:0/457233039 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1cac0716e0 msgr2=0x7f1cac071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:53.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.344+0000 7f1cab400640 1 --2- 172.21.15.40:0/457233039 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1cac0716e0 0x7f1cac071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.344+0000 7f1cab400640 1 -- 172.21.15.40:0/457233039 >> v1:172.21.15.53:6789/0 conn(0x7f1cac0b2970 legacy=0x7f1cac06e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:53.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.344+0000 7f1cab400640 1 -- 172.21.15.40:0/457233039 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0ae0d0 con 0x7f1cac147040 2024-03-08T06:04:53.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.344+0000 7f1cabe00640 1 --2- 172.21.15.40:0/457233039 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1cac0716e0 0x7f1cac071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:53.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.345+0000 7f1cab400640 1 --2- 172.21.15.40:0/457233039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cac147040 0x7f1cac15e1a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0098a0 tx=0x7f1c9c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162d3a010dc19fb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:53.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.346+0000 7f1caaa00640 1 -- 172.21.15.40:0/457233039 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c035020 con 0x7f1cac147040 2024-03-08T06:04:53.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.346+0000 7f1caaa00640 1 -- 172.21.15.40:0/457233039 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c031640 con 0x7f1cac147040 2024-03-08T06:04:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.346+0000 7f1caaa00640 1 -- 172.21.15.40:0/457233039 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c02d040 con 0x7f1cac147040 2024-03-08T06:04:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.346+0000 7f1cb0c00640 1 -- 172.21.15.40:0/457233039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cac147040 msgr2=0x7f1cac15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.346+0000 7f1cb0c00640 1 --2- 172.21.15.40:0/457233039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cac147040 0x7f1cac15e1a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0098a0 tx=0x7f1c9c02b8f0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.347+0000 7f1cb0c00640 1 -- 172.21.15.40:0/457233039 shutdown_connections 2024-03-08T06:04:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.347+0000 7f1cb0c00640 1 --2- 172.21.15.40:0/457233039 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1cac0716e0 0x7f1cac071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.347+0000 7f1cb0c00640 1 --2- 172.21.15.40:0/457233039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cac147040 0x7f1cac15e1a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.347+0000 7f1cb0c00640 1 -- 172.21.15.40:0/457233039 >> 172.21.15.40:0/457233039 conn(0x7f1cac0b1850 msgr2=0x7f1cac0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.347+0000 7f1cb0c00640 1 -- 172.21.15.40:0/457233039 shutdown_connections 2024-03-08T06:04:53.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.347+0000 7f1cb0c00640 1 -- 172.21.15.40:0/457233039 wait complete. 2024-03-08T06:04:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.348+0000 7f1cb0c00640 1 Processor -- start 2024-03-08T06:04:53.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.349+0000 7f1cb0c00640 1 -- start start 2024-03-08T06:04:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.349+0000 7f1cb0c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1cac0716e0 0x7f1cac15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.349+0000 7f1cb0c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cac0b2970 0x7f1cac1e8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1cb0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cac147040 0x7f1cac1e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1cb0c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0ae770 con 0x7f1cac0b2970 2024-03-08T06:04:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1cb0c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0ae5f0 con 0x7f1cac147040 2024-03-08T06:04:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1cb0c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0ae8f0 con 0x7f1cac0716e0 2024-03-08T06:04:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1ca3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cac0b2970 0x7f1cac1e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1ca3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cac0b2970 0x7f1cac1e8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44242/0 (socket says 172.21.15.40:44242) 2024-03-08T06:04:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1ca3e00640 1 -- 172.21.15.40:0/2664165099 learned_addr learned my addr 172.21.15.40:0/2664165099 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1cabe00640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cac147040 0x7f1cac1e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1ca3e00640 1 -- 172.21.15.40:0/2664165099 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1cac0716e0 msgr2=0x7f1cac15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1ca3e00640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1cac0716e0 0x7f1cac15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1ca3e00640 1 -- 172.21.15.40:0/2664165099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cac147040 msgr2=0x7f1cac1e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1ca3e00640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cac147040 0x7f1cac1e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1ca3e00640 1 -- 172.21.15.40:0/2664165099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0afa60 con 0x7f1cac0b2970 2024-03-08T06:04:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.350+0000 7f1cabe00640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cac147040 0x7f1cac1e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.351+0000 7f1ca3e00640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cac0b2970 0x7f1cac1e8610 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c430 tx=0x7f1c9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.351+0000 7f1ca3400640 1 -- 172.21.15.40:0/2664165099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94011790 con 0x7f1cac0b2970 2024-03-08T06:04:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.351+0000 7f1ca3400640 1 -- 172.21.15.40:0/2664165099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c94017070 con 0x7f1cac0b2970 2024-03-08T06:04:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.351+0000 7f1cb0c00640 1 -- 172.21.15.40:0/2664165099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac0afd40 con 0x7f1cac0b2970 2024-03-08T06:04:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.351+0000 7f1ca3400640 1 -- 172.21.15.40:0/2664165099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94011bb0 con 0x7f1cac0b2970 2024-03-08T06:04:53.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.351+0000 7f1cb0c00640 1 -- 172.21.15.40:0/2664165099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac0b0300 con 0x7f1cac0b2970 2024-03-08T06:04:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.352+0000 7f1cb0c00640 1 -- 172.21.15.40:0/2664165099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac075090 con 0x7f1cac0b2970 2024-03-08T06:04:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.353+0000 7f1ca3400640 1 -- 172.21.15.40:0/2664165099 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c9401c460 con 0x7f1cac0b2970 2024-03-08T06:04:53.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.354+0000 7f1ca3400640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c8806b4d0 0x7f1c8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.355+0000 7f1ca3400640 1 -- 172.21.15.40:0/2664165099 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1c9408a0e0 con 0x7f1cac0b2970 2024-03-08T06:04:53.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.355+0000 7f1cab400640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c8806b4d0 0x7f1c8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:53.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.356+0000 7f1cab400640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c8806b4d0 0x7f1c8806d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c036000 tx=0x7f1c9c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:53.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.359+0000 7f1ca3400640 1 -- 172.21.15.40:0/2664165099 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c94058f70 con 0x7f1cac0b2970 2024-03-08T06:04:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.562+0000 7f1cb0c00640 1 -- 172.21.15.40:0/2664165099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1cac0b1380 con 0x7f1cac0b2970 2024-03-08T06:04:53.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.563+0000 7f1ca3400640 1 -- 172.21.15.40:0/2664165099 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c94058870 con 0x7f1cac0b2970 2024-03-08T06:04:53.565 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.566+0000 7f1cb0c00640 1 -- 172.21.15.40:0/2664165099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c8806b4d0 msgr2=0x7f1c8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.566+0000 7f1cb0c00640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c8806b4d0 0x7f1c8806d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c036000 tx=0x7f1c9c030000 comp rx=0 tx=0).stop 2024-03-08T06:04:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.566+0000 7f1cb0c00640 1 -- 172.21.15.40:0/2664165099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cac0b2970 msgr2=0x7f1cac1e8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.566+0000 7f1cb0c00640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cac0b2970 0x7f1cac1e8610 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c430 tx=0x7f1c9400c8f0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.566+0000 7f1cb0c00640 1 -- 172.21.15.40:0/2664165099 shutdown_connections 2024-03-08T06:04:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.566+0000 7f1cb0c00640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1c8806b4d0 0x7f1c8806d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.566+0000 7f1cb0c00640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cac147040 0x7f1cac1e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.566+0000 7f1cb0c00640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cac0b2970 0x7f1cac1e8610 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.566+0000 7f1cb0c00640 1 --2- 172.21.15.40:0/2664165099 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1cac0716e0 0x7f1cac15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.566+0000 7f1cb0c00640 1 -- 172.21.15.40:0/2664165099 >> 172.21.15.40:0/2664165099 conn(0x7f1cac0b1850 msgr2=0x7f1cac0b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.567+0000 7f1cb0c00640 1 -- 172.21.15.40:0/2664165099 shutdown_connections 2024-03-08T06:04:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.567+0000 7f1cb0c00640 1 -- 172.21.15.40:0/2664165099 wait complete. 2024-03-08T06:04:53.587 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:04:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.786+0000 7f28cf200640 1 Processor -- start 2024-03-08T06:04:53.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.787+0000 7f28cf200640 1 -- start start 2024-03-08T06:04:53.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.788+0000 7f28cf200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f28d006f050 0x7f28d0071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.788+0000 7f28cf200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d006d600 con 0x7f28d01451b0 2024-03-08T06:04:53.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.788+0000 7f28cf200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d006d8e0 con 0x7f28d0071940 2024-03-08T06:04:53.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.788+0000 7f28cf200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d0071db0 con 0x7f28d006f050 2024-03-08T06:04:53.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.788+0000 7f28cde00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f28d01451b0 0x7f28d015ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40172/0 (socket says 172.21.15.40:40172) 2024-03-08T06:04:53.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.788+0000 7f28cde00640 1 -- 172.21.15.40:0/1617023523 learned_addr learned my addr 172.21.15.40:0/1617023523 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:53.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28c5400640 1 --2- 172.21.15.40:0/1617023523 >> v2:172.21.15.53:3300/0 conn(0x7f28d006f050 0x7f28d0071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422923934 0 0) 0x7f28d006d600 con 0x7f28d01451b0 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b4003520 con 0x7f28d01451b0 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3603457230 0 0) 0x7f28b4003520 con 0x7f28d01451b0 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 >> v2:172.21.15.53:3300/0 conn(0x7f28d006f050 msgr2=0x7f28d0071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 --2- 172.21.15.40:0/1617023523 >> v2:172.21.15.53:3300/0 conn(0x7f28d006f050 0x7f28d0071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 >> v1:172.21.15.53:6789/0 conn(0x7f28d0071940 legacy=0x7f28d015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d006dbc0 con 0x7f28d01451b0 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f28c0001670 con 0x7f28d01451b0 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 >> v1:172.21.15.40:6789/0 conn(0x7f28d01451b0 legacy=0x7f28d015ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 --2- 172.21.15.40:0/1617023523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 0x7f28b4003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 --2- 172.21.15.40:0/1617023523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b4004800 0x7f28b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.789+0000 7f28cd400640 1 --2- 172.21.15.40:0/1617023523 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f28b40052e0 0x7f28b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d0071db0 con 0x7f28b4003a90 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b400c1e0 con 0x7f28b4004800 2024-03-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b400c3e0 con 0x7f28b40052e0 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28cde00640 1 --2- 172.21.15.40:0/1617023523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 0x7f28b4003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28c5400640 1 --2- 172.21.15.40:0/1617023523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b4004800 0x7f28b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28cde00640 1 -- 172.21.15.40:0/1617023523 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f28b40052e0 msgr2=0x7f28b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28cde00640 1 --2- 172.21.15.40:0/1617023523 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f28b40052e0 0x7f28b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28cde00640 1 -- 172.21.15.40:0/1617023523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b4004800 msgr2=0x7f28b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28cde00640 1 --2- 172.21.15.40:0/1617023523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b4004800 0x7f28b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28cde00640 1 -- 172.21.15.40:0/1617023523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b4003520 con 0x7f28b4003a90 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28ce800640 1 -- 172.21.15.40:0/1617023523 reap_dead start 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.790+0000 7f28c5400640 1 --2- 172.21.15.40:0/1617023523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b4004800 0x7f28b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.791+0000 7f28cde00640 1 --2- 172.21.15.40:0/1617023523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 0x7f28b4003e90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c210 tx=0x7f28b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303f3fc54feb3461 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.791+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0007d30 con 0x7f28b4003a90 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.791+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0030070 con 0x7f28b4003a90 2024-03-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.791+0000 7f28cd400640 1 -- 172.21.15.40:0/1617023523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0005d70 con 0x7f28b4003a90 2024-03-08T06:04:53.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.792+0000 7f28cf200640 1 -- 172.21.15.40:0/1617023523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 msgr2=0x7f28b4003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:53.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.792+0000 7f28cf200640 1 --2- 172.21.15.40:0/1617023523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 0x7f28b4003e90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c210 tx=0x7f28b4005830 comp rx=0 tx=0).stop 2024-03-08T06:04:53.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.792+0000 7f28cf200640 1 -- 172.21.15.40:0/1617023523 shutdown_connections 2024-03-08T06:04:53.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.792+0000 7f28cf200640 1 --2- 172.21.15.40:0/1617023523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 0x7f28b4003e90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.792+0000 7f28cf200640 1 -- 172.21.15.40:0/1617023523 >> 172.21.15.40:0/1617023523 conn(0x7f28d00b1590 msgr2=0x7f28d00b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:53.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.792+0000 7f28cf200640 1 -- 172.21.15.40:0/1617023523 shutdown_connections 2024-03-08T06:04:53.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.792+0000 7f28cf200640 1 -- 172.21.15.40:0/1617023523 wait complete. 2024-03-08T06:04:53.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.794+0000 7f28cf200640 1 Processor -- start 2024-03-08T06:04:53.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.794+0000 7f28cf200640 1 -- start start 2024-03-08T06:04:53.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.795+0000 7f28cf200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 0x7f28d015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.795+0000 7f28cf200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d00c5860 0x7f28d00c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.795+0000 7f28cf200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f28d00c5ce0 0x7f28d00c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.795+0000 7f28cf200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d0145710 con 0x7f28b4003a90 2024-03-08T06:04:53.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.795+0000 7f28cf200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d0145590 con 0x7f28d00c5860 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.795+0000 7f28cf200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d0071400 con 0x7f28d00c5ce0 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.796+0000 7f28cde00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 0x7f28d015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.796+0000 7f28cde00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 0x7f28d015b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44262/0 (socket says 172.21.15.40:44262) 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.796+0000 7f28cde00640 1 -- 172.21.15.40:0/1726846304 learned_addr learned my addr 172.21.15.40:0/1726846304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.796+0000 7f28c5400640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d00c5860 0x7f28d00c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.796+0000 7f28cde00640 1 -- 172.21.15.40:0/1726846304 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f28d00c5ce0 msgr2=0x7f28d00c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.796+0000 7f28cde00640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f28d00c5ce0 0x7f28d00c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.796+0000 7f28cde00640 1 -- 172.21.15.40:0/1726846304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d00c5860 msgr2=0x7f28d00c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.796+0000 7f28cde00640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d00c5860 0x7f28d00c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.796+0000 7f28cde00640 1 -- 172.21.15.40:0/1726846304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d015e430 con 0x7f28b4003a90 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.796+0000 7f28c5400640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d00c5860 0x7f28d00c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.796+0000 7f28cde00640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 0x7f28d015b460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c330 tx=0x7f28c002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.797+0000 7f28c6a00640 1 -- 172.21.15.40:0/1726846304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0007d30 con 0x7f28b4003a90 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.797+0000 7f28c6a00640 1 -- 172.21.15.40:0/1726846304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0004bf0 con 0x7f28b4003a90 2024-03-08T06:04:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.797+0000 7f28c6a00640 1 -- 172.21.15.40:0/1726846304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0007070 con 0x7f28b4003a90 2024-03-08T06:04:53.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.797+0000 7f28cf200640 1 -- 172.21.15.40:0/1726846304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d015e6f0 con 0x7f28b4003a90 2024-03-08T06:04:53.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.797+0000 7f28cf200640 1 -- 172.21.15.40:0/1726846304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d015ecb0 con 0x7f28b4003a90 2024-03-08T06:04:53.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.798+0000 7f28cf200640 1 -- 172.21.15.40:0/1726846304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d01439a0 con 0x7f28b4003a90 2024-03-08T06:04:53.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.799+0000 7f28c6a00640 1 -- 172.21.15.40:0/1726846304 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28c0048020 con 0x7f28b4003a90 2024-03-08T06:04:53.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.799+0000 7f28c6a00640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f28ac06b4d0 0x7f28ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:53.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.800+0000 7f28c6a00640 1 -- 172.21.15.40:0/1726846304 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f28c00ab590 con 0x7f28b4003a90 2024-03-08T06:04:53.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.800+0000 7f28c5400640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f28ac06b4d0 0x7f28ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:53.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.800+0000 7f28c5400640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f28ac06b4d0 0x7f28ac06d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28d00c5360 tx=0x7f28b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:53.804+0000 7f28c6a00640 1 -- 172.21.15.40:0/1726846304 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28c007a380 con 0x7f28b4003a90 2024-03-08T06:04:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.015+0000 7f28cf200640 1 -- 172.21.15.40:0/1726846304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f28d0075090 con 0x7f28b4003a90 2024-03-08T06:04:54.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.016+0000 7f28c6a00640 1 -- 172.21.15.40:0/1726846304 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f28c0080080 con 0x7f28b4003a90 2024-03-08T06:04:54.017 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-03-08T06:04:54.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.018+0000 7f28cf200640 1 -- 172.21.15.40:0/1726846304 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f28ac06b4d0 msgr2=0x7f28ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:54.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.018+0000 7f28cf200640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f28ac06b4d0 0x7f28ac06d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28d00c5360 tx=0x7f28b8008040 comp rx=0 tx=0).stop 2024-03-08T06:04:54.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.018+0000 7f28cf200640 1 -- 172.21.15.40:0/1726846304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 msgr2=0x7f28d015b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:54.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.018+0000 7f28cf200640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 0x7f28d015b460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c330 tx=0x7f28c002e070 comp rx=0 tx=0).stop 2024-03-08T06:04:54.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.019+0000 7f28cf200640 1 -- 172.21.15.40:0/1726846304 shutdown_connections 2024-03-08T06:04:54.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.019+0000 7f28cf200640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f28ac06b4d0 0x7f28ac06d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.019+0000 7f28cf200640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f28d00c5ce0 0x7f28d00c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.019+0000 7f28cf200640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d00c5860 0x7f28d00c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.019+0000 7f28cf200640 1 --2- 172.21.15.40:0/1726846304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b4003a90 0x7f28d015b460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.019+0000 7f28cf200640 1 -- 172.21.15.40:0/1726846304 >> 172.21.15.40:0/1726846304 conn(0x7f28d00b1590 msgr2=0x7f28d00ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.019+0000 7f28cf200640 1 -- 172.21.15.40:0/1726846304 shutdown_connections 2024-03-08T06:04:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.019+0000 7f28cf200640 1 -- 172.21.15.40:0/1726846304 wait complete. 2024-03-08T06:04:54.039 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-08T06:04:54.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:54.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-08T06:04:54.155 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:04:54.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:04:54.155 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-08T06:04:54.212 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-08T06:04:54.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:04:54.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.434+0000 7f9706800640 1 Processor -- start 2024-03-08T06:04:54.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.434+0000 7f9706800640 1 -- start start 2024-03-08T06:04:54.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9706800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f970015bea0 0x7f970015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9706800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9700071df0 0x7f970006d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9706800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970015ceb0 con 0x7f9700145520 2024-03-08T06:04:54.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9706800640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970015c330 con 0x7f9700071df0 2024-03-08T06:04:54.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9706800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970015c610 con 0x7f970015bea0 2024-03-08T06:04:54.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9705e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9700071df0 0x7f970006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9705e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9700071df0 0x7f970006d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44276/0 (socket says 172.21.15.40:44276) 2024-03-08T06:04:54.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9705e00640 1 -- 172.21.15.40:0/4130407156 learned_addr learned my addr 172.21.15.40:0/4130407156 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:54.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9705e00640 1 -- 172.21.15.40:0/4130407156 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f970015bea0 msgr2=0x7f970015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:54.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9704a00640 1 --2- 172.21.15.40:0/4130407156 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f970015bea0 0x7f970015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9705e00640 1 --2- 172.21.15.40:0/4130407156 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f970015bea0 0x7f970015d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.436+0000 7f9705e00640 1 -- 172.21.15.40:0/4130407156 >> v1:172.21.15.53:6789/0 conn(0x7f9700145520 legacy=0x7f97000716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:54.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.437+0000 7f9705e00640 1 -- 172.21.15.40:0/4130407156 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970015c8f0 con 0x7f9700071df0 2024-03-08T06:04:54.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.438+0000 7f9705e00640 1 --2- 172.21.15.40:0/4130407156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9700071df0 0x7f970006d570 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009e50 tx=0x7f96f802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03037a21a6bc356 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:54.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.438+0000 7f96ffe00640 1 -- 172.21.15.40:0/4130407156 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f802ca70 con 0x7f9700071df0 2024-03-08T06:04:54.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.438+0000 7f96ffe00640 1 -- 172.21.15.40:0/4130407156 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f802cc10 con 0x7f9700071df0 2024-03-08T06:04:54.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.438+0000 7f96ffe00640 1 -- 172.21.15.40:0/4130407156 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80343f0 con 0x7f9700071df0 2024-03-08T06:04:54.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.439+0000 7f9706800640 1 -- 172.21.15.40:0/4130407156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9700071df0 msgr2=0x7f970006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:54.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.439+0000 7f9706800640 1 --2- 172.21.15.40:0/4130407156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9700071df0 0x7f970006d570 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009e50 tx=0x7f96f802bed0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.439+0000 7f9706800640 1 -- 172.21.15.40:0/4130407156 shutdown_connections 2024-03-08T06:04:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.439+0000 7f9706800640 1 --2- 172.21.15.40:0/4130407156 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f970015bea0 0x7f970015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.439+0000 7f9706800640 1 --2- 172.21.15.40:0/4130407156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9700071df0 0x7f970006d570 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.439+0000 7f9706800640 1 -- 172.21.15.40:0/4130407156 >> 172.21.15.40:0/4130407156 conn(0x7f97000b1850 msgr2=0x7f97000b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.439+0000 7f9706800640 1 -- 172.21.15.40:0/4130407156 shutdown_connections 2024-03-08T06:04:54.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.439+0000 7f9706800640 1 -- 172.21.15.40:0/4130407156 wait complete. 2024-03-08T06:04:54.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.441+0000 7f9706800640 1 Processor -- start 2024-03-08T06:04:54.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.441+0000 7f9706800640 1 -- start start 2024-03-08T06:04:54.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.442+0000 7f9706800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9700071df0 0x7f97000b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.442+0000 7f9706800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9700145520 0x7f970013fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.442+0000 7f9706800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97001e5310 0x7f97001411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9706800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000af1c0 con 0x7f9700145520 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9706800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000af040 con 0x7f97001e5310 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9706800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970006e410 con 0x7f9700071df0 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9704a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9700145520 0x7f970013fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9704a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9700145520 0x7f970013fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44280/0 (socket says 172.21.15.40:44280) 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9704a00640 1 -- 172.21.15.40:0/2772987577 learned_addr learned my addr 172.21.15.40:0/2772987577 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9705400640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9700071df0 0x7f97000b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9704a00640 1 -- 172.21.15.40:0/2772987577 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9700071df0 msgr2=0x7f97000b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9705e00640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97001e5310 0x7f97001411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9704a00640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9700071df0 0x7f97000b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9704a00640 1 -- 172.21.15.40:0/2772987577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97001e5310 msgr2=0x7f97001411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9704a00640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97001e5310 0x7f97001411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9704a00640 1 -- 172.21.15.40:0/2772987577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97000b0960 con 0x7f9700145520 2024-03-08T06:04:54.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.443+0000 7f9705e00640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97001e5310 0x7f97001411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:54.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.444+0000 7f9704a00640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9700145520 0x7f970013fdd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c7f0 tx=0x7f96ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.444+0000 7f96fd600640 1 -- 172.21.15.40:0/2772987577 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec015070 con 0x7f9700145520 2024-03-08T06:04:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.444+0000 7f96fd600640 1 -- 172.21.15.40:0/2772987577 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec004220 con 0x7f9700145520 2024-03-08T06:04:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.444+0000 7f96fd600640 1 -- 172.21.15.40:0/2772987577 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec007620 con 0x7f9700145520 2024-03-08T06:04:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.444+0000 7f9706800640 1 -- 172.21.15.40:0/2772987577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97001e57d0 con 0x7f9700145520 2024-03-08T06:04:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.444+0000 7f9706800640 1 -- 172.21.15.40:0/2772987577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97001e5d90 con 0x7f9700145520 2024-03-08T06:04:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.446+0000 7f9706800640 1 -- 172.21.15.40:0/2772987577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97001377e0 con 0x7f9700145520 2024-03-08T06:04:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.446+0000 7f96fd600640 1 -- 172.21.15.40:0/2772987577 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96ec0079d0 con 0x7f9700145520 2024-03-08T06:04:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.448+0000 7f96fd600640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96e006b4d0 0x7f96e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.448+0000 7f96fd600640 1 -- 172.21.15.40:0/2772987577 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f96ec08a650 con 0x7f9700145520 2024-03-08T06:04:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.449+0000 7f9705400640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96e006b4d0 0x7f96e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.449+0000 7f9705400640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96e006b4d0 0x7f96e006d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f96f40015d0 tx=0x7f96f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.454+0000 7f96fd600640 1 -- 172.21.15.40:0/2772987577 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97001377e0 con 0x7f9700145520 2024-03-08T06:04:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.697+0000 7f9706800640 1 -- 172.21.15.40:0/2772987577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97000cf0c0 con 0x7f9700145520 2024-03-08T06:04:54.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.698+0000 7f96fd600640 1 -- 172.21.15.40:0/2772987577 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f96ec0593b0 con 0x7f9700145520 2024-03-08T06:04:54.700 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:54.700 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3266388900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3266388900},{"type":"v1","addr":"172.21.15.40:6837","nonce":3266388900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2471668569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2471668569},{"type":"v1","addr":"172.21.15.53:6834","nonce":2471668569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3262228631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3262228631},{"type":"v1","addr":"172.21.15.53:6835","nonce":3262228631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:04:02.003553+0000","modified":"2024-03-08T06:04:05.045865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3709744112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3709744112},{"type":"v1","addr":"172.21.15.40:6835","nonce":3709744112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:04:54.700 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-03-08T06:04:54.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.701+0000 7f9706800640 1 -- 172.21.15.40:0/2772987577 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96e006b4d0 msgr2=0x7f96e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:54.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.701+0000 7f9706800640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96e006b4d0 0x7f96e006d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f96f40015d0 tx=0x7f96f4009040 comp rx=0 tx=0).stop 2024-03-08T06:04:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.702+0000 7f9706800640 1 -- 172.21.15.40:0/2772987577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9700145520 msgr2=0x7f970013fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.702+0000 7f9706800640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9700145520 0x7f970013fdd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c7f0 tx=0x7f96ec00ccb0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.702+0000 7f9706800640 1 -- 172.21.15.40:0/2772987577 shutdown_connections 2024-03-08T06:04:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.702+0000 7f9706800640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f96e006b4d0 0x7f96e006d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.702+0000 7f9706800640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97001e5310 0x7f97001411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.702+0000 7f9706800640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9700145520 0x7f970013fdd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.702+0000 7f9706800640 1 --2- 172.21.15.40:0/2772987577 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9700071df0 0x7f97000b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.702+0000 7f9706800640 1 -- 172.21.15.40:0/2772987577 >> 172.21.15.40:0/2772987577 conn(0x7f97000b1850 msgr2=0x7f970015eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:54.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.702+0000 7f9706800640 1 -- 172.21.15.40:0/2772987577 shutdown_connections 2024-03-08T06:04:54.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.702+0000 7f9706800640 1 -- 172.21.15.40:0/2772987577 wait complete. 2024-03-08T06:04:54.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.939+0000 7ff99e400640 1 Processor -- start 2024-03-08T06:04:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.939+0000 7ff99e400640 1 -- start start 2024-03-08T06:04:54.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.940+0000 7ff99e400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff99806e2f0 0x7ff99806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.940+0000 7ff99e400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff99806f070 0x7ff9980cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.940+0000 7ff99e400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9980720a0 con 0x7ff998144840 2024-03-08T06:04:54.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.941+0000 7ff99e400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff998144c20 con 0x7ff99806f070 2024-03-08T06:04:54.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.941+0000 7ff99e400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99806f4a0 con 0x7ff99806e2f0 2024-03-08T06:04:54.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.941+0000 7ff997e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff99806f070 0x7ff9980cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.941+0000 7ff997e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff99806f070 0x7ff9980cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44284/0 (socket says 172.21.15.40:44284) 2024-03-08T06:04:54.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.941+0000 7ff997e00640 1 -- 172.21.15.40:0/3819338075 learned_addr learned my addr 172.21.15.40:0/3819338075 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:54.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.941+0000 7ff98fe00640 1 --2- 172.21.15.40:0/3819338075 >> v2:172.21.15.53:3300/0 conn(0x7ff99806e2f0 0x7ff99806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.942+0000 7ff997e00640 1 -- 172.21.15.40:0/3819338075 >> v2:172.21.15.53:3300/0 conn(0x7ff99806e2f0 msgr2=0x7ff99806eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:54.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.942+0000 7ff997e00640 1 --2- 172.21.15.40:0/3819338075 >> v2:172.21.15.53:3300/0 conn(0x7ff99806e2f0 0x7ff99806eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.942+0000 7ff997e00640 1 -- 172.21.15.40:0/3819338075 >> v1:172.21.15.53:6789/0 conn(0x7ff998144840 legacy=0x7ff998071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:54.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.942+0000 7ff997e00640 1 -- 172.21.15.40:0/3819338075 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99806d810 con 0x7ff99806f070 2024-03-08T06:04:54.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.943+0000 7ff997e00640 1 --2- 172.21.15.40:0/3819338075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff99806f070 0x7ff9980cc160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff998145810 tx=0x7ff99002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c213c87a3ba634 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:54.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.943+0000 7ff996a00640 1 -- 172.21.15.40:0/3819338075 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99002cb20 con 0x7ff99806f070 2024-03-08T06:04:54.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.943+0000 7ff996a00640 1 -- 172.21.15.40:0/3819338075 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99002ccc0 con 0x7ff99806f070 2024-03-08T06:04:54.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.943+0000 7ff996a00640 1 -- 172.21.15.40:0/3819338075 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990034610 con 0x7ff99806f070 2024-03-08T06:04:54.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.944+0000 7ff99e400640 1 -- 172.21.15.40:0/3819338075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff99806f070 msgr2=0x7ff9980cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:54.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.944+0000 7ff99e400640 1 --2- 172.21.15.40:0/3819338075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff99806f070 0x7ff9980cc160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff998145810 tx=0x7ff99002bf80 comp rx=0 tx=0).stop 2024-03-08T06:04:54.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.944+0000 7ff99e400640 1 -- 172.21.15.40:0/3819338075 shutdown_connections 2024-03-08T06:04:54.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.944+0000 7ff99e400640 1 --2- 172.21.15.40:0/3819338075 >> v2:172.21.15.53:3300/0 conn(0x7ff99806e2f0 0x7ff99806eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.944+0000 7ff99e400640 1 --2- 172.21.15.40:0/3819338075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff99806f070 0x7ff9980cc160 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.944+0000 7ff99e400640 1 -- 172.21.15.40:0/3819338075 >> 172.21.15.40:0/3819338075 conn(0x7ff99806ac50 msgr2=0x7ff99806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:54.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.944+0000 7ff99e400640 1 -- 172.21.15.40:0/3819338075 shutdown_connections 2024-03-08T06:04:54.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.944+0000 7ff99e400640 1 -- 172.21.15.40:0/3819338075 wait complete. 2024-03-08T06:04:54.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.946+0000 7ff99e400640 1 Processor -- start 2024-03-08T06:04:54.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.946+0000 7ff99e400640 1 -- start start 2024-03-08T06:04:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.947+0000 7ff99e400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff99806e2f0 0x7ff9980b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.947+0000 7ff99e400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff99806f070 0x7ff9981f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.947+0000 7ff99e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff998144840 0x7ff99813bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.947+0000 7ff99e400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff998145010 con 0x7ff99806e2f0 2024-03-08T06:04:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.947+0000 7ff99e400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99806ddd0 con 0x7ff998144840 2024-03-08T06:04:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.947+0000 7ff99e400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff998071d10 con 0x7ff99806f070 2024-03-08T06:04:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.947+0000 7ff997400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff99806e2f0 0x7ff9980b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.947+0000 7ff997400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff99806e2f0 0x7ff9980b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44296/0 (socket says 172.21.15.40:44296) 2024-03-08T06:04:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.947+0000 7ff997400640 1 -- 172.21.15.40:0/497124665 learned_addr learned my addr 172.21.15.40:0/497124665 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:54.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.948+0000 7ff997e00640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff998144840 0x7ff99813bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.948+0000 7ff98fe00640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff99806f070 0x7ff9981f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.948+0000 7ff997400640 1 -- 172.21.15.40:0/497124665 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff99806f070 msgr2=0x7ff9981f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.948+0000 7ff997400640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff99806f070 0x7ff9981f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.948+0000 7ff997400640 1 -- 172.21.15.40:0/497124665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff998144840 msgr2=0x7ff99813bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.948+0000 7ff997400640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff998144840 0x7ff99813bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.948+0000 7ff997400640 1 -- 172.21.15.40:0/497124665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9980b3ad0 con 0x7ff99806e2f0 2024-03-08T06:04:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.948+0000 7ff997400640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff99806e2f0 0x7ff9980b3440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c400 tx=0x7ff98800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.949+0000 7ff98f400640 1 -- 172.21.15.40:0/497124665 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988017070 con 0x7ff99806e2f0 2024-03-08T06:04:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.949+0000 7ff98f400640 1 -- 172.21.15.40:0/497124665 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9880052d0 con 0x7ff99806e2f0 2024-03-08T06:04:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.949+0000 7ff98f400640 1 -- 172.21.15.40:0/497124665 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988003830 con 0x7ff99806e2f0 2024-03-08T06:04:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.949+0000 7ff99e400640 1 -- 172.21.15.40:0/497124665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9980b3db0 con 0x7ff99806e2f0 2024-03-08T06:04:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.949+0000 7ff99e400640 1 -- 172.21.15.40:0/497124665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff998137230 con 0x7ff99806e2f0 2024-03-08T06:04:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.950+0000 7ff99e400640 1 -- 172.21.15.40:0/497124665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff998067cb0 con 0x7ff99806e2f0 2024-03-08T06:04:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.951+0000 7ff98f400640 1 -- 172.21.15.40:0/497124665 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff98801a070 con 0x7ff99806e2f0 2024-03-08T06:04:54.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.952+0000 7ff98f400640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff97406b4d0 0x7ff97406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:54.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.952+0000 7ff98f400640 1 -- 172.21.15.40:0/497124665 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff988089550 con 0x7ff99806e2f0 2024-03-08T06:04:54.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.952+0000 7ff98fe00640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff97406b4d0 0x7ff97406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.953+0000 7ff98fe00640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff97406b4d0 0x7ff97406d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff98000a970 tx=0x7ff980008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:54.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:54.957+0000 7ff98f400640 1 -- 172.21.15.40:0/497124665 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9880583e0 con 0x7ff99806e2f0 2024-03-08T06:04:55.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.157+0000 7ff99e400640 1 -- 172.21.15.40:0/497124665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff99813eb30 con 0x7ff99806e2f0 2024-03-08T06:04:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.159+0000 7ff98f400640 1 -- 172.21.15.40:0/497124665 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16770 (secure 0 0 0) 0x7ff988087e20 con 0x7ff99806e2f0 2024-03-08T06:04:55.161 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:55.161 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:48.785806+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 -- 172.21.15.40:0/497124665 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff97406b4d0 msgr2=0x7ff97406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff97406b4d0 0x7ff97406d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff98000a970 tx=0x7ff980008040 comp rx=0 tx=0).stop 2024-03-08T06:04:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 -- 172.21.15.40:0/497124665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff99806e2f0 msgr2=0x7ff9980b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff99806e2f0 0x7ff9980b3440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c400 tx=0x7ff98800c8c0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 -- 172.21.15.40:0/497124665 shutdown_connections 2024-03-08T06:04:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff97406b4d0 0x7ff97406d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff998144840 0x7ff99813bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff99806f070 0x7ff9981f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 --2- 172.21.15.40:0/497124665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff99806e2f0 0x7ff9980b3440 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 -- 172.21.15.40:0/497124665 >> 172.21.15.40:0/497124665 conn(0x7ff99806ac50 msgr2=0x7ff998142330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 -- 172.21.15.40:0/497124665 shutdown_connections 2024-03-08T06:04:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.162+0000 7ff99e400640 1 -- 172.21.15.40:0/497124665 wait complete. 2024-03-08T06:04:55.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:03:59.826823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:04:01.016829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:04:55.183 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:04:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.390+0000 7efe31200640 1 Processor -- start 2024-03-08T06:04:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.390+0000 7efe31200640 1 -- start start 2024-03-08T06:04:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.392+0000 7efe31200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efe2c0b42c0 0x7efe2c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.392+0000 7efe31200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efe2c06e870 0x7efe2c15e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.392+0000 7efe31200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe2c0cd3e0 con 0x7efe2c144240 2024-03-08T06:04:55.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.392+0000 7efe31200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe2c0b3e30 con 0x7efe2c06e870 2024-03-08T06:04:55.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.392+0000 7efe31200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe2c0cd6c0 con 0x7efe2c0b42c0 2024-03-08T06:04:55.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.392+0000 7efe2be00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efe2c06e870 0x7efe2c15e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:55.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.393+0000 7efe2be00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efe2c06e870 0x7efe2c15e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44306/0 (socket says 172.21.15.40:44306) 2024-03-08T06:04:55.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.393+0000 7efe2be00640 1 -- 172.21.15.40:0/218489084 learned_addr learned my addr 172.21.15.40:0/218489084 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:55.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.393+0000 7efe23e00640 1 --2- 172.21.15.40:0/218489084 >> v2:172.21.15.53:3300/0 conn(0x7efe2c0b42c0 0x7efe2c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:55.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.393+0000 7efe2be00640 1 -- 172.21.15.40:0/218489084 >> v2:172.21.15.53:3300/0 conn(0x7efe2c0b42c0 msgr2=0x7efe2c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:55.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.393+0000 7efe2be00640 1 --2- 172.21.15.40:0/218489084 >> v2:172.21.15.53:3300/0 conn(0x7efe2c0b42c0 0x7efe2c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.393+0000 7efe2be00640 1 -- 172.21.15.40:0/218489084 >> v1:172.21.15.53:6789/0 conn(0x7efe2c144240 legacy=0x7efe2c0cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:55.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.393+0000 7efe2be00640 1 -- 172.21.15.40:0/218489084 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c0b37e0 con 0x7efe2c06e870 2024-03-08T06:04:55.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.394+0000 7efe2be00640 1 --2- 172.21.15.40:0/218489084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe2c06e870 0x7efe2c15e460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efe2c15e3c0 tx=0x7efe2402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4765552b7e0e94b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:55.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.394+0000 7efe2aa00640 1 -- 172.21.15.40:0/218489084 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2402ca80 con 0x7efe2c06e870 2024-03-08T06:04:55.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.394+0000 7efe2aa00640 1 -- 172.21.15.40:0/218489084 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2402cc20 con 0x7efe2c06e870 2024-03-08T06:04:55.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.395+0000 7efe2aa00640 1 -- 172.21.15.40:0/218489084 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240343f0 con 0x7efe2c06e870 2024-03-08T06:04:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.395+0000 7efe31200640 1 -- 172.21.15.40:0/218489084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe2c06e870 msgr2=0x7efe2c15e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.395+0000 7efe31200640 1 --2- 172.21.15.40:0/218489084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe2c06e870 0x7efe2c15e460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efe2c15e3c0 tx=0x7efe2402bee0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.395+0000 7efe31200640 1 -- 172.21.15.40:0/218489084 shutdown_connections 2024-03-08T06:04:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.395+0000 7efe31200640 1 --2- 172.21.15.40:0/218489084 >> v2:172.21.15.53:3300/0 conn(0x7efe2c0b42c0 0x7efe2c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.395+0000 7efe31200640 1 --2- 172.21.15.40:0/218489084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe2c06e870 0x7efe2c15e460 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.395+0000 7efe31200640 1 -- 172.21.15.40:0/218489084 >> 172.21.15.40:0/218489084 conn(0x7efe2c0b1850 msgr2=0x7efe2c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.395+0000 7efe31200640 1 -- 172.21.15.40:0/218489084 shutdown_connections 2024-03-08T06:04:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.395+0000 7efe31200640 1 -- 172.21.15.40:0/218489084 wait complete. 2024-03-08T06:04:55.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.397+0000 7efe31200640 1 Processor -- start 2024-03-08T06:04:55.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.397+0000 7efe31200640 1 -- start start 2024-03-08T06:04:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.398+0000 7efe31200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe2c06e870 0x7efe2c13b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.398+0000 7efe31200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe2c144240 0x7efe2c162530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.398+0000 7efe31200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe2c06d8e0 0x7efe2c06dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.398+0000 7efe31200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c144620 con 0x7efe2c06e870 2024-03-08T06:04:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.398+0000 7efe31200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c0cd0c0 con 0x7efe2c06d8e0 2024-03-08T06:04:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.398+0000 7efe31200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c0b4b30 con 0x7efe2c144240 2024-03-08T06:04:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.398+0000 7efe2b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe2c06e870 0x7efe2c13b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.398+0000 7efe2b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe2c06e870 0x7efe2c13b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44322/0 (socket says 172.21.15.40:44322) 2024-03-08T06:04:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.398+0000 7efe2b400640 1 -- 172.21.15.40:0/4104850097 learned_addr learned my addr 172.21.15.40:0/4104850097 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe2be00640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe2c06d8e0 0x7efe2c06dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe23e00640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe2c144240 0x7efe2c162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe2b400640 1 -- 172.21.15.40:0/4104850097 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe2c144240 msgr2=0x7efe2c162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe2b400640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe2c144240 0x7efe2c162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe2b400640 1 -- 172.21.15.40:0/4104850097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe2c06d8e0 msgr2=0x7efe2c06dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe2b400640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe2c06d8e0 0x7efe2c06dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe2b400640 1 -- 172.21.15.40:0/4104850097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c165c00 con 0x7efe2c06e870 2024-03-08T06:04:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe23e00640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe2c144240 0x7efe2c162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe2b400640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe2c06e870 0x7efe2c13b130 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00b420 tx=0x7efe1c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe23400640 1 -- 172.21.15.40:0/4104850097 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c011830 con 0x7efe2c06e870 2024-03-08T06:04:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe23400640 1 -- 172.21.15.40:0/4104850097 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c016070 con 0x7efe2c06e870 2024-03-08T06:04:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.399+0000 7efe23400640 1 -- 172.21.15.40:0/4104850097 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c011c10 con 0x7efe2c06e870 2024-03-08T06:04:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.400+0000 7efe31200640 1 -- 172.21.15.40:0/4104850097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2c13b980 con 0x7efe2c06e870 2024-03-08T06:04:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.400+0000 7efe31200640 1 -- 172.21.15.40:0/4104850097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2c166180 con 0x7efe2c06e870 2024-03-08T06:04:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.401+0000 7efe31200640 1 -- 172.21.15.40:0/4104850097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2c075090 con 0x7efe2c06e870 2024-03-08T06:04:55.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.402+0000 7efe23400640 1 -- 172.21.15.40:0/4104850097 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe1c01b460 con 0x7efe2c06e870 2024-03-08T06:04:55.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.402+0000 7efe23400640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe0806b500 0x7efe0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.403+0000 7efe23400640 1 -- 172.21.15.40:0/4104850097 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efe1c089d00 con 0x7efe2c06e870 2024-03-08T06:04:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.403+0000 7efe23e00640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe0806b500 0x7efe0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.404+0000 7efe23e00640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe0806b500 0x7efe0806d9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efe2c06e520 tx=0x7efe14008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:55.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.407+0000 7efe23400640 1 -- 172.21.15.40:0/4104850097 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe1c020020 con 0x7efe2c06e870 2024-03-08T06:04:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.610+0000 7efe31200640 1 -- 172.21.15.40:0/4104850097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe2c06f160 con 0x7efe2c06e870 2024-03-08T06:04:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.611+0000 7efe23400640 1 -- 172.21.15.40:0/4104850097 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe1c058b00 con 0x7efe2c06e870 2024-03-08T06:04:55.612 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:04:55.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 -- 172.21.15.40:0/4104850097 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe0806b500 msgr2=0x7efe0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:55.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe0806b500 0x7efe0806d9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efe2c06e520 tx=0x7efe14008040 comp rx=0 tx=0).stop 2024-03-08T06:04:55.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 -- 172.21.15.40:0/4104850097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe2c06e870 msgr2=0x7efe2c13b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:55.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe2c06e870 0x7efe2c13b130 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00b420 tx=0x7efe1c00b8e0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 -- 172.21.15.40:0/4104850097 shutdown_connections 2024-03-08T06:04:55.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe0806b500 0x7efe0806d9c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe2c06d8e0 0x7efe2c06dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe2c144240 0x7efe2c162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 --2- 172.21.15.40:0/4104850097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe2c06e870 0x7efe2c13b130 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 -- 172.21.15.40:0/4104850097 >> 172.21.15.40:0/4104850097 conn(0x7efe2c0b1850 msgr2=0x7efe2c06ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:55.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 -- 172.21.15.40:0/4104850097 shutdown_connections 2024-03-08T06:04:55.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.614+0000 7efe31200640 1 -- 172.21.15.40:0/4104850097 wait complete. 2024-03-08T06:04:55.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:04:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.863+0000 7fa214600640 1 Processor -- start 2024-03-08T06:04:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.863+0000 7fa214600640 1 -- start start 2024-03-08T06:04:55.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.864+0000 7fa214600640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa20c071fa0 0x7fa20c0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa214600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa20c15e760 0x7fa20c144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa214600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20c15ebf0 con 0x7fa20c071bc0 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa214600640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20c15ee70 con 0x7fa20c071fa0 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa214600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c15f150 con 0x7fa20c15e760 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa211200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa20c071fa0 0x7fa20c0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa211200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa20c071fa0 0x7fa20c0cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44334/0 (socket says 172.21.15.40:44334) 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa211200640 1 -- 172.21.15.40:0/1423339704 learned_addr learned my addr 172.21.15.40:0/1423339704 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa211c00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa20c071bc0 0x7fa20c15b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40182/0 (socket says 172.21.15.40:40182) 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa212600640 1 --2- 172.21.15.40:0/1423339704 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa20c15e760 0x7fa20c144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa211200640 1 -- 172.21.15.40:0/1423339704 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa20c15e760 msgr2=0x7fa20c144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa211200640 1 --2- 172.21.15.40:0/1423339704 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa20c15e760 0x7fa20c144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.865+0000 7fa211200640 1 -- 172.21.15.40:0/1423339704 >> v1:172.21.15.40:6789/0 conn(0x7fa20c071bc0 legacy=0x7fa20c15b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.866+0000 7fa211200640 1 -- 172.21.15.40:0/1423339704 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c15f430 con 0x7fa20c071fa0 2024-03-08T06:04:55.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.866+0000 7fa20be00640 1 -- 172.21.15.40:0/1423339704 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601203577 0 0) 0x7fa200001110 con 0x7fa20c071bc0 2024-03-08T06:04:55.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.867+0000 7fa211200640 1 --2- 172.21.15.40:0/1423339704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa20c071fa0 0x7fa20c0cf5b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009870 tx=0x7fa1f802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f06b6f867a7f95 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:55.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.867+0000 7fa20be00640 1 -- 172.21.15.40:0/1423339704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8035020 con 0x7fa20c071fa0 2024-03-08T06:04:55.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.867+0000 7fa20be00640 1 -- 172.21.15.40:0/1423339704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f8031640 con 0x7fa20c071fa0 2024-03-08T06:04:55.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.867+0000 7fa20be00640 1 -- 172.21.15.40:0/1423339704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f802d040 con 0x7fa20c071fa0 2024-03-08T06:04:55.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.868+0000 7fa214600640 1 -- 172.21.15.40:0/1423339704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa20c071fa0 msgr2=0x7fa20c0cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:55.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.868+0000 7fa214600640 1 --2- 172.21.15.40:0/1423339704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa20c071fa0 0x7fa20c0cf5b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009870 tx=0x7fa1f802b8f0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.868+0000 7fa214600640 1 -- 172.21.15.40:0/1423339704 shutdown_connections 2024-03-08T06:04:55.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.868+0000 7fa214600640 1 --2- 172.21.15.40:0/1423339704 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa20c15e760 0x7fa20c144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.868+0000 7fa214600640 1 --2- 172.21.15.40:0/1423339704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa20c071fa0 0x7fa20c0cf5b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.868+0000 7fa214600640 1 -- 172.21.15.40:0/1423339704 >> 172.21.15.40:0/1423339704 conn(0x7fa20c06ae30 msgr2=0x7fa20c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:55.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.868+0000 7fa214600640 1 -- 172.21.15.40:0/1423339704 shutdown_connections 2024-03-08T06:04:55.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.868+0000 7fa214600640 1 -- 172.21.15.40:0/1423339704 wait complete. 2024-03-08T06:04:55.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.870+0000 7fa214600640 1 Processor -- start 2024-03-08T06:04:55.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.870+0000 7fa214600640 1 -- start start 2024-03-08T06:04:55.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.871+0000 7fa214600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa20c071bc0 0x7fa20c0cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.871+0000 7fa214600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa20c071fa0 0x7fa20c0c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.871+0000 7fa214600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa20c15e760 0x7fa20c15ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.871+0000 7fa214600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c1448d0 con 0x7fa20c071fa0 2024-03-08T06:04:55.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.871+0000 7fa214600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c144750 con 0x7fa20c071bc0 2024-03-08T06:04:55.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.871+0000 7fa214600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0b3fd0 con 0x7fa20c15e760 2024-03-08T06:04:55.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.871+0000 7fa211200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa20c071fa0 0x7fa20c0c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:55.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.871+0000 7fa211200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa20c071fa0 0x7fa20c0c4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44344/0 (socket says 172.21.15.40:44344) 2024-03-08T06:04:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.871+0000 7fa211200640 1 -- 172.21.15.40:0/692015186 learned_addr learned my addr 172.21.15.40:0/692015186 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa211c00640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa20c071bc0 0x7fa20c0cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa211200640 1 -- 172.21.15.40:0/692015186 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa20c15e760 msgr2=0x7fa20c15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa211200640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa20c15e760 0x7fa20c15ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa211200640 1 -- 172.21.15.40:0/692015186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa20c071bc0 msgr2=0x7fa20c0cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa211200640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa20c071bc0 0x7fa20c0cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa211200640 1 -- 172.21.15.40:0/692015186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0c5160 con 0x7fa20c071fa0 2024-03-08T06:04:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa211c00640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa20c071bc0 0x7fa20c0cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:55.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa211200640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa20c071fa0 0x7fa20c0c4c20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8000c00 tx=0x7fa1f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa209600640 1 -- 172.21.15.40:0/692015186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8040070 con 0x7fa20c071fa0 2024-03-08T06:04:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa209600640 1 -- 172.21.15.40:0/692015186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f80399f0 con 0x7fa20c071fa0 2024-03-08T06:04:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa209600640 1 -- 172.21.15.40:0/692015186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8035040 con 0x7fa20c071fa0 2024-03-08T06:04:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.872+0000 7fa214600640 1 -- 172.21.15.40:0/692015186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20c0c5620 con 0x7fa20c071fa0 2024-03-08T06:04:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.873+0000 7fa214600640 1 -- 172.21.15.40:0/692015186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c0c5be0 con 0x7fa20c071fa0 2024-03-08T06:04:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.874+0000 7fa214600640 1 -- 172.21.15.40:0/692015186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c067cb0 con 0x7fa20c071fa0 2024-03-08T06:04:55.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.875+0000 7fa209600640 1 -- 172.21.15.40:0/692015186 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1f804b060 con 0x7fa20c071fa0 2024-03-08T06:04:55.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.876+0000 7fa209600640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa1ec06b4d0 0x7fa1ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:55.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.876+0000 7fa209600640 1 -- 172.21.15.40:0/692015186 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa1f80abc20 con 0x7fa20c071fa0 2024-03-08T06:04:55.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.876+0000 7fa211c00640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa1ec06b4d0 0x7fa1ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:55.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.877+0000 7fa211c00640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa1ec06b4d0 0x7fa1ec06d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa2000013b0 tx=0x7fa200009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:55.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:55.881+0000 7fa209600640 1 -- 172.21.15.40:0/692015186 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1f807aa20 con 0x7fa20c071fa0 2024-03-08T06:04:56.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.081+0000 7fa214600640 1 -- 172.21.15.40:0/692015186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa20c0b2d90 con 0x7fa20c071fa0 2024-03-08T06:04:56.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.082+0000 7fa209600640 1 -- 172.21.15.40:0/692015186 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1f807a320 con 0x7fa20c071fa0 2024-03-08T06:04:56.083 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:04:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.084+0000 7fa214600640 1 -- 172.21.15.40:0/692015186 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa1ec06b4d0 msgr2=0x7fa1ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.085+0000 7fa214600640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa1ec06b4d0 0x7fa1ec06d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa2000013b0 tx=0x7fa200009040 comp rx=0 tx=0).stop 2024-03-08T06:04:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.085+0000 7fa214600640 1 -- 172.21.15.40:0/692015186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa20c071fa0 msgr2=0x7fa20c0c4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.085+0000 7fa214600640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa20c071fa0 0x7fa20c0c4c20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8000c00 tx=0x7fa1f80046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.085+0000 7fa214600640 1 -- 172.21.15.40:0/692015186 shutdown_connections 2024-03-08T06:04:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.085+0000 7fa214600640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa1ec06b4d0 0x7fa1ec06d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.085+0000 7fa214600640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa20c15e760 0x7fa20c15ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.085+0000 7fa214600640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa20c071fa0 0x7fa20c0c4c20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.085+0000 7fa214600640 1 --2- 172.21.15.40:0/692015186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa20c071bc0 0x7fa20c0cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.085+0000 7fa214600640 1 -- 172.21.15.40:0/692015186 >> 172.21.15.40:0/692015186 conn(0x7fa20c06ae30 msgr2=0x7fa20c074510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.085+0000 7fa214600640 1 -- 172.21.15.40:0/692015186 shutdown_connections 2024-03-08T06:04:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.085+0000 7fa214600640 1 -- 172.21.15.40:0/692015186 wait complete. 2024-03-08T06:04:56.106 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:04:56.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.334+0000 7feedda00640 1 Processor -- start 2024-03-08T06:04:56.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.334+0000 7feedda00640 1 -- start start 2024-03-08T06:04:56.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.336+0000 7feedda00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feed8145590 0x7feed80cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.336+0000 7feedda00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feed80b3bd0 0x7feed80cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.336+0000 7feedda00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed8071990 con 0x7feed80ccc70 2024-03-08T06:04:56.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.336+0000 7feedda00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed80cd5f0 con 0x7feed80b3bd0 2024-03-08T06:04:56.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.336+0000 7feedda00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed8071c70 con 0x7feed8145590 2024-03-08T06:04:56.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.336+0000 7feed6000640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feed80ccc70 0x7feed815f7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40192/0 (socket says 172.21.15.40:40192) 2024-03-08T06:04:56.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.336+0000 7feed6000640 1 -- 172.21.15.40:0/3377525142 learned_addr learned my addr 172.21.15.40:0/3377525142 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:56.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.336+0000 7feed7400640 1 --2- 172.21.15.40:0/3377525142 >> v2:172.21.15.40:3300/0 conn(0x7feed80b3bd0 0x7feed80cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.336+0000 7feed6a00640 1 --2- 172.21.15.40:0/3377525142 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feed8145590 0x7feed80cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.337+0000 7feed5600640 1 -- 172.21.15.40:0/3377525142 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924865917 0 0) 0x7feed8071990 con 0x7feed80ccc70 2024-03-08T06:04:56.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.337+0000 7feed5600640 1 -- 172.21.15.40:0/3377525142 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feebc003520 con 0x7feed80ccc70 2024-03-08T06:04:56.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.337+0000 7feed7400640 1 -- 172.21.15.40:0/3377525142 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feed8145590 msgr2=0x7feed80cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:56.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.337+0000 7feed7400640 1 --2- 172.21.15.40:0/3377525142 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feed8145590 0x7feed80cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.337+0000 7feed7400640 1 -- 172.21.15.40:0/3377525142 >> v1:172.21.15.40:6789/0 conn(0x7feed80ccc70 legacy=0x7feed815f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:56.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.337+0000 7feed7400640 1 -- 172.21.15.40:0/3377525142 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed8071f50 con 0x7feed80b3bd0 2024-03-08T06:04:56.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.338+0000 7feed7400640 1 --2- 172.21.15.40:0/3377525142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed80b3bd0 0x7feed80cd0b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feed00097a0 tx=0x7feed002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a31c90fac48c227 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:56.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.338+0000 7feed5600640 1 -- 172.21.15.40:0/3377525142 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0035020 con 0x7feed80b3bd0 2024-03-08T06:04:56.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.338+0000 7feed5600640 1 -- 172.21.15.40:0/3377525142 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed0031810 con 0x7feed80b3bd0 2024-03-08T06:04:56.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.338+0000 7feed5600640 1 -- 172.21.15.40:0/3377525142 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0031bc0 con 0x7feed80b3bd0 2024-03-08T06:04:56.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.339+0000 7feedda00640 1 -- 172.21.15.40:0/3377525142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed80b3bd0 msgr2=0x7feed80cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:56.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.339+0000 7feedda00640 1 --2- 172.21.15.40:0/3377525142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed80b3bd0 0x7feed80cd0b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feed00097a0 tx=0x7feed002b8b0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.339+0000 7feedda00640 1 -- 172.21.15.40:0/3377525142 shutdown_connections 2024-03-08T06:04:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.339+0000 7feedda00640 1 --2- 172.21.15.40:0/3377525142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed80b3bd0 0x7feed80cd0b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.339+0000 7feedda00640 1 --2- 172.21.15.40:0/3377525142 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feed8145590 0x7feed80cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.339+0000 7feedda00640 1 -- 172.21.15.40:0/3377525142 >> 172.21.15.40:0/3377525142 conn(0x7feed80cfaa0 msgr2=0x7feed80cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.339+0000 7feedda00640 1 -- 172.21.15.40:0/3377525142 shutdown_connections 2024-03-08T06:04:56.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.339+0000 7feedda00640 1 -- 172.21.15.40:0/3377525142 wait complete. 2024-03-08T06:04:56.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.341+0000 7feedda00640 1 Processor -- start 2024-03-08T06:04:56.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.341+0000 7feedda00640 1 -- start start 2024-03-08T06:04:56.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.342+0000 7feedda00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed80b3bd0 0x7feed8155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.342+0000 7feedda00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feed80ccc70 0x7feed8142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.342+0000 7feedda00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed8145590 0x7feed8142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.342+0000 7feedda00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8145970 con 0x7feed80b3bd0 2024-03-08T06:04:56.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.342+0000 7feedda00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8072510 con 0x7feed8145590 2024-03-08T06:04:56.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.342+0000 7feedda00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed80cd960 con 0x7feed80ccc70 2024-03-08T06:04:56.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.342+0000 7feed6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed80b3bd0 0x7feed8155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.342+0000 7feed6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed80b3bd0 0x7feed8155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44358/0 (socket says 172.21.15.40:44358) 2024-03-08T06:04:56.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.342+0000 7feed6a00640 1 -- 172.21.15.40:0/3407373938 learned_addr learned my addr 172.21.15.40:0/3407373938 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:56.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.343+0000 7feed6a00640 1 -- 172.21.15.40:0/3407373938 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feed80ccc70 msgr2=0x7feed8142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:56.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.343+0000 7feed6000640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feed80ccc70 0x7feed8142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.343+0000 7feed6a00640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feed80ccc70 0x7feed8142130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.343+0000 7feed6a00640 1 -- 172.21.15.40:0/3407373938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed8145590 msgr2=0x7feed8142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:56.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.343+0000 7feed7400640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed8145590 0x7feed8142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.343+0000 7feed6a00640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed8145590 0x7feed8142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.343+0000 7feed6a00640 1 -- 172.21.15.40:0/3407373938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed815ed90 con 0x7feed80b3bd0 2024-03-08T06:04:56.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.343+0000 7feed7400640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed8145590 0x7feed8142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:56.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.343+0000 7feed6a00640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed80b3bd0 0x7feed8155e70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feec8007ef0 tx=0x7feec800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:56.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.343+0000 7feecea00640 1 -- 172.21.15.40:0/3407373938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8014620 con 0x7feed80b3bd0 2024-03-08T06:04:56.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.344+0000 7feecea00640 1 -- 172.21.15.40:0/3407373938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec8019070 con 0x7feed80b3bd0 2024-03-08T06:04:56.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.344+0000 7feecea00640 1 -- 172.21.15.40:0/3407373938 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8014a00 con 0x7feed80b3bd0 2024-03-08T06:04:56.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.344+0000 7feedda00640 1 -- 172.21.15.40:0/3407373938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed815f070 con 0x7feed80b3bd0 2024-03-08T06:04:56.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.344+0000 7feedda00640 1 -- 172.21.15.40:0/3407373938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed815f630 con 0x7feed80b3bd0 2024-03-08T06:04:56.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.345+0000 7feedda00640 1 -- 172.21.15.40:0/3407373938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed8067cb0 con 0x7feed80b3bd0 2024-03-08T06:04:56.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.346+0000 7feecea00640 1 -- 172.21.15.40:0/3407373938 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feec8014ba0 con 0x7feed80b3bd0 2024-03-08T06:04:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.347+0000 7feecea00640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7feeb406b500 0x7feeb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.347+0000 7feecea00640 1 -- 172.21.15.40:0/3407373938 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7feec808ab00 con 0x7feed80b3bd0 2024-03-08T06:04:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.347+0000 7feed6000640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7feeb406b500 0x7feeb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.348+0000 7feed6000640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7feeb406b500 0x7feeb406d9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feed81f6fb0 tx=0x7feec0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.352+0000 7feecea00640 1 -- 172.21.15.40:0/3407373938 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feec8059990 con 0x7feed80b3bd0 2024-03-08T06:04:56.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.552+0000 7feedda00640 1 -- 172.21.15.40:0/3407373938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feed8152ce0 con 0x7feed80b3bd0 2024-03-08T06:04:56.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.553+0000 7feecea00640 1 -- 172.21.15.40:0/3407373938 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7feec8059290 con 0x7feed80b3bd0 2024-03-08T06:04:56.554 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:04:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 -- 172.21.15.40:0/3407373938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7feeb406b500 msgr2=0x7feeb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7feeb406b500 0x7feeb406d9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feed81f6fb0 tx=0x7feec0009040 comp rx=0 tx=0).stop 2024-03-08T06:04:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 -- 172.21.15.40:0/3407373938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed80b3bd0 msgr2=0x7feed8155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed80b3bd0 0x7feed8155e70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feec8007ef0 tx=0x7feec800e4b0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 -- 172.21.15.40:0/3407373938 shutdown_connections 2024-03-08T06:04:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7feeb406b500 0x7feeb406d9c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed8145590 0x7feed8142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7feed80ccc70 0x7feed8142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 --2- 172.21.15.40:0/3407373938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed80b3bd0 0x7feed8155e70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 -- 172.21.15.40:0/3407373938 >> 172.21.15.40:0/3407373938 conn(0x7feed80cfaa0 msgr2=0x7feed80cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:56.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 -- 172.21.15.40:0/3407373938 shutdown_connections 2024-03-08T06:04:56.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.556+0000 7feedda00640 1 -- 172.21.15.40:0/3407373938 wait complete. 2024-03-08T06:04:56.577 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:04:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.793+0000 7f08c3c00640 1 Processor -- start 2024-03-08T06:04:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.793+0000 7f08c3c00640 1 -- start start 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.795+0000 7f08c3c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f08bc144030 0x7f08bc0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.795+0000 7f08c3c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08bc15e580 0x7f08bc06e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.795+0000 7f08c3c00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc15eaf0 con 0x7f08bc06ec50 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.795+0000 7f08c3c00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08bc06f0b0 con 0x7f08bc144030 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.795+0000 7f08c3c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0b3350 con 0x7f08bc15e580 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.795+0000 7f08c1200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f08bc144030 0x7f08bc0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.795+0000 7f08c1200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f08bc144030 0x7f08bc0cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44372/0 (socket says 172.21.15.40:44372) 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.795+0000 7f08c1200640 1 -- 172.21.15.40:0/1363191880 learned_addr learned my addr 172.21.15.40:0/1363191880 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.795+0000 7f08bbe00640 1 --2- 172.21.15.40:0/1363191880 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08bc15e580 0x7f08bc06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.796+0000 7f08c1200640 1 -- 172.21.15.40:0/1363191880 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08bc15e580 msgr2=0x7f08bc06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.796+0000 7f08c1200640 1 --2- 172.21.15.40:0/1363191880 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08bc15e580 0x7f08bc06e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.796+0000 7f08c1200640 1 -- 172.21.15.40:0/1363191880 >> v1:172.21.15.53:6789/0 conn(0x7f08bc06ec50 legacy=0x7f08bc071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.796+0000 7f08c1200640 1 -- 172.21.15.40:0/1363191880 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc15edd0 con 0x7f08bc144030 2024-03-08T06:04:56.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.796+0000 7f08bbe00640 1 --2- 172.21.15.40:0/1363191880 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08bc15e580 0x7f08bc06e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:56.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.797+0000 7f08c1200640 1 --2- 172.21.15.40:0/1363191880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08bc144030 0x7f08bc0cf080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009e50 tx=0x7f08ac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d467b1b4788726 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:56.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.797+0000 7f08bb400640 1 -- 172.21.15.40:0/1363191880 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac02ca80 con 0x7f08bc144030 2024-03-08T06:04:56.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.797+0000 7f08bb400640 1 -- 172.21.15.40:0/1363191880 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac02cc20 con 0x7f08bc144030 2024-03-08T06:04:56.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.797+0000 7f08bb400640 1 -- 172.21.15.40:0/1363191880 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0343f0 con 0x7f08bc144030 2024-03-08T06:04:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.798+0000 7f08c3c00640 1 -- 172.21.15.40:0/1363191880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08bc144030 msgr2=0x7f08bc0cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.798+0000 7f08c3c00640 1 --2- 172.21.15.40:0/1363191880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08bc144030 0x7f08bc0cf080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009e50 tx=0x7f08ac02bee0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.798+0000 7f08c3c00640 1 -- 172.21.15.40:0/1363191880 shutdown_connections 2024-03-08T06:04:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.798+0000 7f08c3c00640 1 --2- 172.21.15.40:0/1363191880 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08bc15e580 0x7f08bc06e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.798+0000 7f08c3c00640 1 --2- 172.21.15.40:0/1363191880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08bc144030 0x7f08bc0cf080 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.798+0000 7f08c3c00640 1 -- 172.21.15.40:0/1363191880 >> 172.21.15.40:0/1363191880 conn(0x7f08bc06ae30 msgr2=0x7f08bc06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.798+0000 7f08c3c00640 1 -- 172.21.15.40:0/1363191880 shutdown_connections 2024-03-08T06:04:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.798+0000 7f08c3c00640 1 -- 172.21.15.40:0/1363191880 wait complete. 2024-03-08T06:04:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.800+0000 7f08c3c00640 1 Processor -- start 2024-03-08T06:04:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.800+0000 7f08c3c00640 1 -- start start 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.801+0000 7f08c3c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc06ec50 0x7f08bc15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.801+0000 7f08c3c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08bc144030 0x7f08bc15c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.801+0000 7f08c3c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08bc15e580 0x7f08bc071810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.801+0000 7f08c3c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0cf5c0 con 0x7f08bc144030 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.801+0000 7f08c3c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc15f390 con 0x7f08bc06ec50 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.801+0000 7f08c3c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0cf740 con 0x7f08bc15e580 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.801+0000 7f08bbe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08bc144030 0x7f08bc15c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.801+0000 7f08bbe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08bc144030 0x7f08bc15c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44378/0 (socket says 172.21.15.40:44378) 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.801+0000 7f08bbe00640 1 -- 172.21.15.40:0/836218382 learned_addr learned my addr 172.21.15.40:0/836218382 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.802+0000 7f08c1200640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc06ec50 0x7f08bc15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.802+0000 7f08c1c00640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08bc15e580 0x7f08bc071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.802+0000 7f08bbe00640 1 -- 172.21.15.40:0/836218382 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08bc15e580 msgr2=0x7f08bc071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:56.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.802+0000 7f08bbe00640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08bc15e580 0x7f08bc071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.802+0000 7f08bbe00640 1 -- 172.21.15.40:0/836218382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc06ec50 msgr2=0x7f08bc15a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:56.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.802+0000 7f08bbe00640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc06ec50 0x7f08bc15a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:56.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.802+0000 7f08bbe00640 1 -- 172.21.15.40:0/836218382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc15ca60 con 0x7f08bc144030 2024-03-08T06:04:56.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.802+0000 7f08bbe00640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08bc144030 0x7f08bc15c130 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c7f0 tx=0x7f08a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:56.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.802+0000 7f08b8c00640 1 -- 172.21.15.40:0/836218382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8015070 con 0x7f08bc144030 2024-03-08T06:04:56.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.802+0000 7f08b8c00640 1 -- 172.21.15.40:0/836218382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a8007820 con 0x7f08bc144030 2024-03-08T06:04:56.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.802+0000 7f08b8c00640 1 -- 172.21.15.40:0/836218382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8007bd0 con 0x7f08bc144030 2024-03-08T06:04:56.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.803+0000 7f08c3c00640 1 -- 172.21.15.40:0/836218382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc15cd40 con 0x7f08bc144030 2024-03-08T06:04:56.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.803+0000 7f08c3c00640 1 -- 172.21.15.40:0/836218382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc15d300 con 0x7f08bc144030 2024-03-08T06:04:56.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.804+0000 7f08c3c00640 1 -- 172.21.15.40:0/836218382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc067cb0 con 0x7f08bc144030 2024-03-08T06:04:56.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.805+0000 7f08b8c00640 1 -- 172.21.15.40:0/836218382 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08a8018070 con 0x7f08bc144030 2024-03-08T06:04:56.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.805+0000 7f08b8c00640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f089c06b4d0 0x7f089c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:56.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.806+0000 7f08b8c00640 1 -- 172.21.15.40:0/836218382 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f08a808a490 con 0x7f08bc144030 2024-03-08T06:04:56.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.806+0000 7f08c1200640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f089c06b4d0 0x7f089c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:56.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.806+0000 7f08c1200640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f089c06b4d0 0x7f089c06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f08ac035000 tx=0x7f08ac030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:56.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:56.810+0000 7f08b8c00640 1 -- 172.21.15.40:0/836218382 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08a80591f0 con 0x7f08bc144030 2024-03-08T06:04:57.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.047+0000 7f08c3c00640 1 -- 172.21.15.40:0/836218382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08bc06d700 con 0x7f08bc144030 2024-03-08T06:04:57.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.048+0000 7f08b8c00640 1 -- 172.21.15.40:0/836218382 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f08a8059e20 con 0x7f08bc144030 2024-03-08T06:04:57.049 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:57.050 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3266388900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3266388900},{"type":"v1","addr":"172.21.15.40:6837","nonce":3266388900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2471668569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2471668569},{"type":"v1","addr":"172.21.15.53:6834","nonce":2471668569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3262228631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3262228631},{"type":"v1","addr":"172.21.15.53:6835","nonce":3262228631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:04:02.003553+0000","modified":"2024-03-08T06:04:05.045865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3709744112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3709744112},{"type":"v1","addr":"172.21.15.40:6835","nonce":3709744112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:04:57.050 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-03-08T06:04:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 -- 172.21.15.40:0/836218382 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f089c06b4d0 msgr2=0x7f089c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f089c06b4d0 0x7f089c06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f08ac035000 tx=0x7f08ac030000 comp rx=0 tx=0).stop 2024-03-08T06:04:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 -- 172.21.15.40:0/836218382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08bc144030 msgr2=0x7f08bc15c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08bc144030 0x7f08bc15c130 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c7f0 tx=0x7f08a800ccb0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 -- 172.21.15.40:0/836218382 shutdown_connections 2024-03-08T06:04:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f089c06b4d0 0x7f089c06d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08bc15e580 0x7f08bc071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08bc144030 0x7f08bc15c130 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 --2- 172.21.15.40:0/836218382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc06ec50 0x7f08bc15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 -- 172.21.15.40:0/836218382 >> 172.21.15.40:0/836218382 conn(0x7f08bc06ae30 msgr2=0x7f08bc0ae740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 -- 172.21.15.40:0/836218382 shutdown_connections 2024-03-08T06:04:57.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.051+0000 7f08c3c00640 1 -- 172.21.15.40:0/836218382 wait complete. 2024-03-08T06:04:57.071 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:57.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.291+0000 7fa7ee600640 1 Processor -- start 2024-03-08T06:04:57.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.292+0000 7fa7ee600640 1 -- start start 2024-03-08T06:04:57.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.294+0000 7fa7ee600640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7e815dee0 0x7fa7e80b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.294+0000 7fa7ee600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e80d04c0 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.294+0000 7fa7ee600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e80d0790 con 0x7fa7e80d0070 2024-03-08T06:04:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.294+0000 7fa7ee600640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e815e2c0 con 0x7fa7e815dee0 2024-03-08T06:04:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.294+0000 7fa7e6a00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa7e80b3bb0 0x7fa7e80cf960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48096/0 (socket says 172.21.15.40:48096) 2024-03-08T06:04:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.294+0000 7fa7e6a00640 1 -- 172.21.15.40:0/4208943737 learned_addr learned my addr 172.21.15.40:0/4208943737 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:57.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.294+0000 7fa7e7400640 1 --2- 172.21.15.40:0/4208943737 >> v2:172.21.15.40:3300/0 conn(0x7fa7e815dee0 0x7fa7e80b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.294+0000 7fa7e7400640 1 -- 172.21.15.40:0/4208943737 >> v1:172.21.15.53:6789/0 conn(0x7fa7e80d0070 legacy=0x7fa7e80c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:57.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.294+0000 7fa7e7400640 1 -- 172.21.15.40:0/4208943737 >> v1:172.21.15.40:6789/0 conn(0x7fa7e80b3bb0 legacy=0x7fa7e80cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.295+0000 7fa7e7400640 1 -- 172.21.15.40:0/4208943737 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e80d0a70 con 0x7fa7e815dee0 2024-03-08T06:04:57.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.294+0000 7fa7e6000640 1 -- 172.21.15.40:0/4208943737 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626745616 0 0) 0x7fa7e80d04c0 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.296+0000 7fa7e7400640 1 --2- 172.21.15.40:0/4208943737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e815dee0 0x7fa7e80b3670 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009d90 tx=0x7fa7d802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ed1d874eab90d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.296+0000 7fa7e6000640 1 -- 172.21.15.40:0/4208943737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8039070 con 0x7fa7e815dee0 2024-03-08T06:04:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.296+0000 7fa7e6000640 1 -- 172.21.15.40:0/4208943737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d802cc60 con 0x7fa7e815dee0 2024-03-08T06:04:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.296+0000 7fa7e6000640 1 -- 172.21.15.40:0/4208943737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8034660 con 0x7fa7e815dee0 2024-03-08T06:04:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.297+0000 7fa7ee600640 1 -- 172.21.15.40:0/4208943737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e815dee0 msgr2=0x7fa7e80b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.297+0000 7fa7ee600640 1 --2- 172.21.15.40:0/4208943737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e815dee0 0x7fa7e80b3670 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009d90 tx=0x7fa7d802beb0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.297+0000 7fa7ee600640 1 -- 172.21.15.40:0/4208943737 shutdown_connections 2024-03-08T06:04:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.297+0000 7fa7ee600640 1 --2- 172.21.15.40:0/4208943737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e815dee0 0x7fa7e80b3670 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.297+0000 7fa7ee600640 1 -- 172.21.15.40:0/4208943737 >> 172.21.15.40:0/4208943737 conn(0x7fa7e80d1a90 msgr2=0x7fa7e80d1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.297+0000 7fa7ee600640 1 -- 172.21.15.40:0/4208943737 shutdown_connections 2024-03-08T06:04:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.297+0000 7fa7ee600640 1 -- 172.21.15.40:0/4208943737 wait complete. 2024-03-08T06:04:57.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.299+0000 7fa7ee600640 1 Processor -- start 2024-03-08T06:04:57.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.300+0000 7fa7ee600640 1 -- start start 2024-03-08T06:04:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.301+0000 7fa7ee600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80b3bb0 0x7fa7e8151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.301+0000 7fa7ee600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7e80d0070 0x7fa7e81543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.301+0000 7fa7ee600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7e815dee0 0x7fa7e81548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.301+0000 7fa7ee600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80c6e20 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.301+0000 7fa7ee600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80d1030 con 0x7fa7e80d0070 2024-03-08T06:04:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.301+0000 7fa7ee600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80c79c0 con 0x7fa7e815dee0 2024-03-08T06:04:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.301+0000 7fa7e7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80b3bb0 0x7fa7e8151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.301+0000 7fa7e7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80b3bb0 0x7fa7e8151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53318/0 (socket says 172.21.15.40:53318) 2024-03-08T06:04:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.301+0000 7fa7e7400640 1 -- 172.21.15.40:0/970253777 learned_addr learned my addr 172.21.15.40:0/970253777 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.301+0000 7fa7e6a00640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7e80d0070 0x7fa7e81543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.302+0000 7fa7e7400640 1 -- 172.21.15.40:0/970253777 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7e815dee0 msgr2=0x7fa7e81548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.302+0000 7fa7e7e00640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7e815dee0 0x7fa7e81548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.302+0000 7fa7e7400640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7e815dee0 0x7fa7e81548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.302+0000 7fa7e7400640 1 -- 172.21.15.40:0/970253777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7e80d0070 msgr2=0x7fa7e81543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.302+0000 7fa7e7400640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7e80d0070 0x7fa7e81543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.302+0000 7fa7e7400640 1 -- 172.21.15.40:0/970253777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e80cd190 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.302+0000 7fa7e7400640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80b3bb0 0x7fa7e8151640 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8037040 tx=0x7fa7d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:57.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.302+0000 7fa7df400640 1 -- 172.21.15.40:0/970253777 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8044020 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.302+0000 7fa7df400640 1 -- 172.21.15.40:0/970253777 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d803f670 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.302+0000 7fa7ee600640 1 -- 172.21.15.40:0/970253777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e80cd470 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.302+0000 7fa7df400640 1 -- 172.21.15.40:0/970253777 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8039040 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.303+0000 7fa7ee600640 1 -- 172.21.15.40:0/970253777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e80cda30 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.304+0000 7fa7ee600640 1 -- 172.21.15.40:0/970253777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e80683c0 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.305+0000 7fa7df400640 1 -- 172.21.15.40:0/970253777 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7d8047020 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.306+0000 7fa7df400640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7c406b4d0 0x7fa7c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.306+0000 7fa7df400640 1 -- 172.21.15.40:0/970253777 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa7d80acee0 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.306+0000 7fa7e6a00640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7c406b4d0 0x7fa7c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.307+0000 7fa7e6a00640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7c406b4d0 0x7fa7c406d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80afa90 tx=0x7fa7d0007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:57.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.311+0000 7fa7df400640 1 -- 172.21.15.40:0/970253777 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7d804e030 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.512+0000 7fa7ee600640 1 -- 172.21.15.40:0/970253777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7e80cdc70 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.514+0000 7fa7df400640 1 -- 172.21.15.40:0/970253777 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16770 (secure 0 0 0) 0x7fa7d807bce0 con 0x7fa7e80b3bb0 2024-03-08T06:04:57.515 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:57.515 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:48.785806+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:57.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 -- 172.21.15.40:0/970253777 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7c406b4d0 msgr2=0x7fa7c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7c406b4d0 0x7fa7c406d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80afa90 tx=0x7fa7d0007b50 comp rx=0 tx=0).stop 2024-03-08T06:04:57.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 -- 172.21.15.40:0/970253777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80b3bb0 msgr2=0x7fa7e8151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80b3bb0 0x7fa7e8151640 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8037040 tx=0x7fa7d8004a40 comp rx=0 tx=0).stop 2024-03-08T06:04:57.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 -- 172.21.15.40:0/970253777 shutdown_connections 2024-03-08T06:04:57.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa7c406b4d0 0x7fa7c406d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7e815dee0 0x7fa7e81548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7e80d0070 0x7fa7e81543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 --2- 172.21.15.40:0/970253777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80b3bb0 0x7fa7e8151640 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 -- 172.21.15.40:0/970253777 >> 172.21.15.40:0/970253777 conn(0x7fa7e80d1a90 msgr2=0x7fa7e80b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:57.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 -- 172.21.15.40:0/970253777 shutdown_connections 2024-03-08T06:04:57.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.517+0000 7fa7ee600640 1 -- 172.21.15.40:0/970253777 wait complete. 2024-03-08T06:04:57.537 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-08T06:04:57.537 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:04:57.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.751+0000 7ffb15400640 1 Processor -- start 2024-03-08T06:04:57.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.751+0000 7ffb15400640 1 -- start start 2024-03-08T06:04:57.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.752+0000 7ffb15400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb1006d600 0x7ffb10144d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.752+0000 7ffb15400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1006da80 con 0x7ffb10071ab0 2024-03-08T06:04:57.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.752+0000 7ffb15400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1006dd60 con 0x7ffb10071e90 2024-03-08T06:04:57.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.752+0000 7ffb15400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1006ef40 con 0x7ffb1006d600 2024-03-08T06:04:57.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.752+0000 7ffb0e800640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffb10071ab0 0x7ffb1015cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48100/0 (socket says 172.21.15.40:48100) 2024-03-08T06:04:57.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.752+0000 7ffb0e800640 1 -- 172.21.15.40:0/3601472364 learned_addr learned my addr 172.21.15.40:0/3601472364 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:57.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.752+0000 7ffb0de00640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb1006d600 0x7ffb10144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919817953 0 0) 0x7ffb1006da80 con 0x7ffb10071ab0 2024-03-08T06:04:57.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf0003520 con 0x7ffb10071ab0 2024-03-08T06:04:57.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1367883464 0 0) 0x7ffaf0003520 con 0x7ffb10071ab0 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb1006d600 msgr2=0x7ffb10144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb1006d600 0x7ffb10144d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 >> v1:172.21.15.53:6789/0 conn(0x7ffb10071e90 legacy=0x7ffb1015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1006e040 con 0x7ffb10071ab0 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7ffb00001670 con 0x7ffb10071ab0 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 >> v1:172.21.15.40:6789/0 conn(0x7ffb10071ab0 legacy=0x7ffb1015cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf00039e0 0x7ffaf0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf0004800 0x7ffaf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb1006d600 0x7ffaf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf000ba70 con 0x7ffaf0004800 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf000bc20 con 0x7ffaf00039e0 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf000be20 con 0x7ffb1006d600 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.753+0000 7ffb0de00640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf0004800 0x7ffaf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.754+0000 7ffb0e800640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf00039e0 0x7ffaf0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.754+0000 7ffb0de00640 1 -- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb1006d600 msgr2=0x7ffaf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:57.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.754+0000 7ffb0f200640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb1006d600 0x7ffaf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.754+0000 7ffb0de00640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb1006d600 0x7ffaf00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.754+0000 7ffb0de00640 1 -- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf00039e0 msgr2=0x7ffaf0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:57.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.754+0000 7ffb0de00640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf00039e0 0x7ffaf0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.754+0000 7ffb0de00640 1 -- 172.21.15.40:0/3601472364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf0003520 con 0x7ffaf0004800 2024-03-08T06:04:57.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.754+0000 7ffb0f200640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb1006d600 0x7ffaf00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.755+0000 7ffb0de00640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf0004800 0x7ffaf0004c80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009f90 tx=0x7ffaf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7278732abd6194 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.755+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8039070 con 0x7ffaf0004800 2024-03-08T06:04:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.755+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf802cba0 con 0x7ffaf0004800 2024-03-08T06:04:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.755+0000 7ffb0d400640 1 -- 172.21.15.40:0/3601472364 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8009c10 con 0x7ffaf0004800 2024-03-08T06:04:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.756+0000 7ffb15400640 1 -- 172.21.15.40:0/3601472364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf0004800 msgr2=0x7ffaf0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.756+0000 7ffb15400640 1 --2- 172.21.15.40:0/3601472364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf0004800 0x7ffaf0004c80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009f90 tx=0x7ffaf0005830 comp rx=0 tx=0).stop 2024-03-08T06:04:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.756+0000 7ffb0f200640 1 -- 172.21.15.40:0/3601472364 reap_dead start 2024-03-08T06:04:57.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.756+0000 7ffb15400640 1 -- 172.21.15.40:0/3601472364 shutdown_connections 2024-03-08T06:04:57.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.756+0000 7ffb15400640 1 -- 172.21.15.40:0/3601472364 >> 172.21.15.40:0/3601472364 conn(0x7ffb1006ae30 msgr2=0x7ffb1006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:57.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.756+0000 7ffb15400640 1 -- 172.21.15.40:0/3601472364 shutdown_connections 2024-03-08T06:04:57.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.756+0000 7ffb15400640 1 -- 172.21.15.40:0/3601472364 wait complete. 2024-03-08T06:04:57.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.759+0000 7ffb15400640 1 Processor -- start 2024-03-08T06:04:57.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.759+0000 7ffb15400640 1 -- start start 2024-03-08T06:04:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb15400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf0004800 0x7ffb1013b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb15400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb100b0e50 0x7ffb100af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb15400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10140740 0x7ffb10141200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb15400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb10144d60 con 0x7ffb10140740 2024-03-08T06:04:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb15400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1006e600 con 0x7ffaf0004800 2024-03-08T06:04:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb15400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb10144ee0 con 0x7ffb100b0e50 2024-03-08T06:04:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb0f200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10140740 0x7ffb10141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb0f200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10140740 0x7ffb10141200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53336/0 (socket says 172.21.15.40:53336) 2024-03-08T06:04:57.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb0f200640 1 -- 172.21.15.40:0/3672008115 learned_addr learned my addr 172.21.15.40:0/3672008115 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb0e800640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf0004800 0x7ffb1013b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb0de00640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb100b0e50 0x7ffb100af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb0f200640 1 -- 172.21.15.40:0/3672008115 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb100b0e50 msgr2=0x7ffb100af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb0f200640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb100b0e50 0x7ffb100af580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb0f200640 1 -- 172.21.15.40:0/3672008115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf0004800 msgr2=0x7ffb1013b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb0f200640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf0004800 0x7ffb1013b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb0f200640 1 -- 172.21.15.40:0/3672008115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb100afdb0 con 0x7ffb10140740 2024-03-08T06:04:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb0f200640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10140740 0x7ffb10141200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffb100cf5c0 tx=0x7ffb0800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.760+0000 7ffb06a00640 1 -- 172.21.15.40:0/3672008115 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08007030 con 0x7ffb10140740 2024-03-08T06:04:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.761+0000 7ffb06a00640 1 -- 172.21.15.40:0/3672008115 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb080071f0 con 0x7ffb10140740 2024-03-08T06:04:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.761+0000 7ffb06a00640 1 -- 172.21.15.40:0/3672008115 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080075e0 con 0x7ffb10140740 2024-03-08T06:04:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.761+0000 7ffb15400640 1 -- 172.21.15.40:0/3672008115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb100b0090 con 0x7ffb10140740 2024-03-08T06:04:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.761+0000 7ffb15400640 1 -- 172.21.15.40:0/3672008115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb100b0650 con 0x7ffb10140740 2024-03-08T06:04:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.763+0000 7ffb15400640 1 -- 172.21.15.40:0/3672008115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb10067cb0 con 0x7ffb10140740 2024-03-08T06:04:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.763+0000 7ffb06a00640 1 -- 172.21.15.40:0/3672008115 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb0801b020 con 0x7ffb10140740 2024-03-08T06:04:57.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.764+0000 7ffb06a00640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ffaec06b4d0 0x7ffaec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:57.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.764+0000 7ffb06a00640 1 -- 172.21.15.40:0/3672008115 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ffb08089cc0 con 0x7ffb10140740 2024-03-08T06:04:57.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.765+0000 7ffb0e800640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ffaec06b4d0 0x7ffaec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:57.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.765+0000 7ffb0e800640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ffaec06b4d0 0x7ffaec06d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffb00000c00 tx=0x7ffb00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:57.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:57.769+0000 7ffb06a00640 1 -- 172.21.15.40:0/3672008115 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb08090080 con 0x7ffb10140740 2024-03-08T06:04:57.819 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:04:57.827 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:04:58.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.011+0000 7ffb15400640 1 -- 172.21.15.40:0/3672008115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb1006b520 con 0x7ffb10140740 2024-03-08T06:04:58.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.011+0000 7ffb06a00640 1 -- 172.21.15.40:0/3672008115 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7ffb08058ab0 con 0x7ffb10140740 2024-03-08T06:04:58.012 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:58.012 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3266388900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3266388900},{"type":"v1","addr":"172.21.15.40:6837","nonce":3266388900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2471668569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2471668569},{"type":"v1","addr":"172.21.15.53:6834","nonce":2471668569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3262228631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3262228631},{"type":"v1","addr":"172.21.15.53:6835","nonce":3262228631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:04:02.003553+0000","modified":"2024-03-08T06:04:05.045865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3709744112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3709744112},{"type":"v1","addr":"172.21.15.40:6835","nonce":3709744112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:04:58.013 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-03-08T06:04:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 -- 172.21.15.40:0/3672008115 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ffaec06b4d0 msgr2=0x7ffaec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ffaec06b4d0 0x7ffaec06d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffb00000c00 tx=0x7ffb00009040 comp rx=0 tx=0).stop 2024-03-08T06:04:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 -- 172.21.15.40:0/3672008115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10140740 msgr2=0x7ffb10141200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10140740 0x7ffb10141200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffb100cf5c0 tx=0x7ffb0800d890 comp rx=0 tx=0).stop 2024-03-08T06:04:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 -- 172.21.15.40:0/3672008115 shutdown_connections 2024-03-08T06:04:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ffaec06b4d0 0x7ffaec06d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10140740 0x7ffb10141200 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb100b0e50 0x7ffb100af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 --2- 172.21.15.40:0/3672008115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf0004800 0x7ffb1013b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 -- 172.21.15.40:0/3672008115 >> 172.21.15.40:0/3672008115 conn(0x7ffb1006ae30 msgr2=0x7ffb100b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:58.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 -- 172.21.15.40:0/3672008115 shutdown_connections 2024-03-08T06:04:58.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.014+0000 7ffb15400640 1 -- 172.21.15.40:0/3672008115 wait complete. 2024-03-08T06:04:58.033 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:04:58.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.241+0000 7fc3cee00640 1 Processor -- start 2024-03-08T06:04:58.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.241+0000 7fc3cee00640 1 -- start start 2024-03-08T06:04:58.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cee00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3c815dce0 0x7fc3c80d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cee00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3c80d3690 0x7fc3c80b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cee00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc3c806c290 0x7fc3c806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cee00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c80b9b80 con 0x7fc3c815dce0 2024-03-08T06:04:58.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cee00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c80d25f0 con 0x7fc3c806c290 2024-03-08T06:04:58.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cee00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80d28d0 con 0x7fc3c80d3690 2024-03-08T06:04:58.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cda00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3c815dce0 0x7fc3c80d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:58.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cda00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3c815dce0 0x7fc3c80d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53346/0 (socket says 172.21.15.40:53346) 2024-03-08T06:04:58.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cda00640 1 -- 172.21.15.40:0/145236847 learned_addr learned my addr 172.21.15.40:0/145236847 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:58.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cda00640 1 -- 172.21.15.40:0/145236847 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3c80d3690 msgr2=0x7fc3c80b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:58.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cda00640 1 --2- 172.21.15.40:0/145236847 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3c80d3690 0x7fc3c80b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cda00640 1 -- 172.21.15.40:0/145236847 >> v2:172.21.15.53:3300/0 conn(0x7fc3c806c290 msgr2=0x7fc3c806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:58.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cda00640 1 --2- 172.21.15.40:0/145236847 >> v2:172.21.15.53:3300/0 conn(0x7fc3c806c290 0x7fc3c806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.243+0000 7fc3cda00640 1 -- 172.21.15.40:0/145236847 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c80d2bb0 con 0x7fc3c815dce0 2024-03-08T06:04:58.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.244+0000 7fc3cda00640 1 --2- 172.21.15.40:0/145236847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c815dce0 0x7fc3c80d1fa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80b92b0 tx=0x7fc3bc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201e72fbdda49371 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:58.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.244+0000 7fc3c7e00640 1 -- 172.21.15.40:0/145236847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc034070 con 0x7fc3c815dce0 2024-03-08T06:04:58.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.244+0000 7fc3c7e00640 1 -- 172.21.15.40:0/145236847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc030860 con 0x7fc3c815dce0 2024-03-08T06:04:58.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.244+0000 7fc3c7e00640 1 -- 172.21.15.40:0/145236847 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc030c30 con 0x7fc3c815dce0 2024-03-08T06:04:58.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.245+0000 7fc3cee00640 1 -- 172.21.15.40:0/145236847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c815dce0 msgr2=0x7fc3c80d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.245+0000 7fc3cee00640 1 --2- 172.21.15.40:0/145236847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c815dce0 0x7fc3c80d1fa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80b92b0 tx=0x7fc3bc02b9a0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.245+0000 7fc3cee00640 1 -- 172.21.15.40:0/145236847 shutdown_connections 2024-03-08T06:04:58.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.245+0000 7fc3cee00640 1 --2- 172.21.15.40:0/145236847 >> v2:172.21.15.53:3300/0 conn(0x7fc3c806c290 0x7fc3c806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.245+0000 7fc3cee00640 1 --2- 172.21.15.40:0/145236847 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3c80d3690 0x7fc3c80b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.245+0000 7fc3cee00640 1 --2- 172.21.15.40:0/145236847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c815dce0 0x7fc3c80d1fa0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.245+0000 7fc3cee00640 1 -- 172.21.15.40:0/145236847 >> 172.21.15.40:0/145236847 conn(0x7fc3c8066fc0 msgr2=0x7fc3c80d3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:58.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.245+0000 7fc3cee00640 1 -- 172.21.15.40:0/145236847 shutdown_connections 2024-03-08T06:04:58.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.245+0000 7fc3cee00640 1 -- 172.21.15.40:0/145236847 wait complete. 2024-03-08T06:04:58.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.248+0000 7fc3cee00640 1 Processor -- start 2024-03-08T06:04:58.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.248+0000 7fc3cee00640 1 -- start start 2024-03-08T06:04:58.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cee00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c806c290 0x7fc3c81f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cee00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3c80d3690 0x7fc3c81557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cee00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3c815dce0 0x7fc3c8153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cee00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c815e0c0 con 0x7fc3c806c290 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cee00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80d3170 con 0x7fc3c815dce0 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cee00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c815e240 con 0x7fc3c80d3690 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cda00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c806c290 0x7fc3c81f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cda00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c806c290 0x7fc3c81f2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53350/0 (socket says 172.21.15.40:53350) 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cda00640 1 -- 172.21.15.40:0/2181784386 learned_addr learned my addr 172.21.15.40:0/2181784386 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cda00640 1 -- 172.21.15.40:0/2181784386 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3c80d3690 msgr2=0x7fc3c81557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cda00640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3c80d3690 0x7fc3c81557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cda00640 1 -- 172.21.15.40:0/2181784386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3c815dce0 msgr2=0x7fc3c8153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3ce400640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3c815dce0 0x7fc3c8153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cda00640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3c815dce0 0x7fc3c8153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cda00640 1 -- 172.21.15.40:0/2181784386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c8156a60 con 0x7fc3c806c290 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3ce400640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3c815dce0 0x7fc3c8153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cda00640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c806c290 0x7fc3c81f2f20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc000c00 tx=0x7fc3bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:58.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3c5600640 1 -- 172.21.15.40:0/2181784386 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc034040 con 0x7fc3c806c290 2024-03-08T06:04:58.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3c5600640 1 -- 172.21.15.40:0/2181784386 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc039d90 con 0x7fc3c806c290 2024-03-08T06:04:58.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3c5600640 1 -- 172.21.15.40:0/2181784386 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc043630 con 0x7fc3c806c290 2024-03-08T06:04:58.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.250+0000 7fc3cee00640 1 -- 172.21.15.40:0/2181784386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c81561c0 con 0x7fc3c806c290 2024-03-08T06:04:58.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.251+0000 7fc3cee00640 1 -- 172.21.15.40:0/2181784386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c8156780 con 0x7fc3c806c290 2024-03-08T06:04:58.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.252+0000 7fc3cee00640 1 -- 172.21.15.40:0/2181784386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c8075100 con 0x7fc3c806c290 2024-03-08T06:04:58.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.254+0000 7fc3c5600640 1 -- 172.21.15.40:0/2181784386 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3bc037070 con 0x7fc3c806c290 2024-03-08T06:04:58.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.255+0000 7fc3c5600640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.255+0000 7fc3c5600640 1 -- 172.21.15.40:0/2181784386 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc3bc0ac4d0 con 0x7fc3c806c290 2024-03-08T06:04:58.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.255+0000 7fc3cd000640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:58.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.255+0000 7fc3cd000640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80cdb40 tx=0x7fc3b8006c90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:58.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.258+0000 7fc3c5600640 1 -- 172.21.15.40:0/2181784386 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3bc03e070 con 0x7fc3c806c290 2024-03-08T06:04:58.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.495+0000 7fc3cee00640 1 -- 172.21.15.40:0/2181784386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3c815bd30 con 0x7fc3c806c290 2024-03-08T06:04:58.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.497+0000 7fc3c5600640 1 -- 172.21.15.40:0/2181784386 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fc3bc07b360 con 0x7fc3c806c290 2024-03-08T06:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3266388900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3266388900},{"type":"v1","addr":"172.21.15.40:6837","nonce":3266388900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2471668569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2471668569},{"type":"v1","addr":"172.21.15.53:6834","nonce":2471668569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3262228631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3262228631},{"type":"v1","addr":"172.21.15.53:6835","nonce":3262228631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:04:02.003553+0000","modified":"2024-03-08T06:04:05.045865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3709744112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3709744112},{"type":"v1","addr":"172.21.15.40:6835","nonce":3709744112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T06:04:58.498 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-03-08T06:04:58.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.499+0000 7fc3cee00640 1 -- 172.21.15.40:0/2181784386 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc3ac06b4d0 msgr2=0x7fc3ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.499+0000 7fc3cee00640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80cdb40 tx=0x7fc3b8006c90 comp rx=0 tx=0).stop 2024-03-08T06:04:58.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.499+0000 7fc3cee00640 1 -- 172.21.15.40:0/2181784386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c806c290 msgr2=0x7fc3c81f2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.499+0000 7fc3cee00640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c806c290 0x7fc3c81f2f20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc000c00 tx=0x7fc3bc0046c0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.501+0000 7fc3cee00640 1 -- 172.21.15.40:0/2181784386 shutdown_connections 2024-03-08T06:04:58.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.501+0000 7fc3cee00640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.501+0000 7fc3cee00640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3c815dce0 0x7fc3c8153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.501+0000 7fc3cee00640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3c80d3690 0x7fc3c81557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.501+0000 7fc3cee00640 1 --2- 172.21.15.40:0/2181784386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3c806c290 0x7fc3c81f2f20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.501+0000 7fc3cee00640 1 -- 172.21.15.40:0/2181784386 >> 172.21.15.40:0/2181784386 conn(0x7fc3c8066fc0 msgr2=0x7fc3c80b7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.501+0000 7fc3cee00640 1 -- 172.21.15.40:0/2181784386 shutdown_connections 2024-03-08T06:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.501+0000 7fc3cee00640 1 -- 172.21.15.40:0/2181784386 wait complete. 2024-03-08T06:04:58.519 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:04:58.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.730+0000 7f1d19a00640 1 Processor -- start 2024-03-08T06:04:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.730+0000 7f1d19a00640 1 -- start start 2024-03-08T06:04:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.732+0000 7f1d19a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d14071ab0 0x7f1d1415cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.732+0000 7f1d19a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d14071e90 0x7f1d1415d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.732+0000 7f1d19a00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1415d0c0 con 0x7f1d1406d600 2024-03-08T06:04:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.732+0000 7f1d19a00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1406ef40 con 0x7f1d14071ab0 2024-03-08T06:04:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.732+0000 7f1d19a00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1406da80 con 0x7f1d14071e90 2024-03-08T06:04:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.733+0000 7f1d12a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d14071ab0 0x7f1d1415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.733+0000 7f1d12000640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1d1406d600 0x7f1d14144d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48108/0 (socket says 172.21.15.40:48108) 2024-03-08T06:04:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.733+0000 7f1d12000640 1 -- 172.21.15.40:0/4184590249 learned_addr learned my addr 172.21.15.40:0/4184590249 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.733+0000 7f1d13400640 1 --2- 172.21.15.40:0/4184590249 >> v2:172.21.15.53:3300/0 conn(0x7f1d14071e90 0x7f1d1415d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.733+0000 7f1d11600640 1 -- 172.21.15.40:0/4184590249 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617061206 0 0) 0x7f1d00001110 con 0x7f1d1406d600 2024-03-08T06:04:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.733+0000 7f1d12a00640 1 -- 172.21.15.40:0/4184590249 >> v2:172.21.15.53:3300/0 conn(0x7f1d14071e90 msgr2=0x7f1d1415d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.733+0000 7f1d12a00640 1 --2- 172.21.15.40:0/4184590249 >> v2:172.21.15.53:3300/0 conn(0x7f1d14071e90 0x7f1d1415d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.733+0000 7f1d12a00640 1 -- 172.21.15.40:0/4184590249 >> v1:172.21.15.40:6789/0 conn(0x7f1d1406d600 legacy=0x7f1d14144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.733+0000 7f1d12a00640 1 -- 172.21.15.40:0/4184590249 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1406dd60 con 0x7f1d14071ab0 2024-03-08T06:04:58.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.734+0000 7f1d12a00640 1 --2- 172.21.15.40:0/4184590249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d14071ab0 0x7f1d1415cb80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1d140cf2e0 tx=0x7f1d0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce803830cdd4b579 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:58.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.735+0000 7f1d11600640 1 -- 172.21.15.40:0/4184590249 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04035020 con 0x7f1d14071ab0 2024-03-08T06:04:58.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.735+0000 7f1d11600640 1 -- 172.21.15.40:0/4184590249 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d04031830 con 0x7f1d14071ab0 2024-03-08T06:04:58.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.735+0000 7f1d11600640 1 -- 172.21.15.40:0/4184590249 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04031c00 con 0x7f1d14071ab0 2024-03-08T06:04:58.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.735+0000 7f1d19a00640 1 -- 172.21.15.40:0/4184590249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d14071ab0 msgr2=0x7f1d1415cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.735+0000 7f1d19a00640 1 --2- 172.21.15.40:0/4184590249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d14071ab0 0x7f1d1415cb80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1d140cf2e0 tx=0x7f1d0402b8d0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.735+0000 7f1d19a00640 1 -- 172.21.15.40:0/4184590249 shutdown_connections 2024-03-08T06:04:58.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.735+0000 7f1d19a00640 1 --2- 172.21.15.40:0/4184590249 >> v2:172.21.15.53:3300/0 conn(0x7f1d14071e90 0x7f1d1415d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.735+0000 7f1d19a00640 1 --2- 172.21.15.40:0/4184590249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d14071ab0 0x7f1d1415cb80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.735+0000 7f1d19a00640 1 -- 172.21.15.40:0/4184590249 >> 172.21.15.40:0/4184590249 conn(0x7f1d1406ae30 msgr2=0x7f1d1406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:58.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.736+0000 7f1d19a00640 1 -- 172.21.15.40:0/4184590249 shutdown_connections 2024-03-08T06:04:58.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.736+0000 7f1d19a00640 1 -- 172.21.15.40:0/4184590249 wait complete. 2024-03-08T06:04:58.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.737+0000 7f1d19a00640 1 Processor -- start 2024-03-08T06:04:58.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.738+0000 7f1d19a00640 1 -- start start 2024-03-08T06:04:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d19a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1406d600 0x7f1d1415b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d19a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d14071ab0 0x7f1d141e5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d19a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1d14071e90 0x7f1d140cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d19a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1406e600 con 0x7f1d14071ab0 2024-03-08T06:04:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d19a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1406e320 con 0x7f1d1406d600 2024-03-08T06:04:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d19a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14145470 con 0x7f1d14071e90 2024-03-08T06:04:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d12000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d14071ab0 0x7f1d141e5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d12000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d14071ab0 0x7f1d141e5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53368/0 (socket says 172.21.15.40:53368) 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d12000640 1 -- 172.21.15.40:0/2181326891 learned_addr learned my addr 172.21.15.40:0/2181326891 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d12a00640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1406d600 0x7f1d1415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d12000640 1 -- 172.21.15.40:0/2181326891 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1d14071e90 msgr2=0x7f1d140cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d12000640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1d14071e90 0x7f1d140cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d12000640 1 -- 172.21.15.40:0/2181326891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1406d600 msgr2=0x7f1d1415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d12000640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1406d600 0x7f1d1415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.739+0000 7f1d12000640 1 -- 172.21.15.40:0/2181326891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d141e65b0 con 0x7f1d14071ab0 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.740+0000 7f1d12000640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d14071ab0 0x7f1d141e5530 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000c7b0 tx=0x7f1d0000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.740+0000 7f1d0aa00640 1 -- 172.21.15.40:0/2181326891 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00018070 con 0x7f1d14071ab0 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.740+0000 7f1d0aa00640 1 -- 172.21.15.40:0/2181326891 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d000070d0 con 0x7f1d14071ab0 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.740+0000 7f1d19a00640 1 -- 172.21.15.40:0/2181326891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d141e5ff0 con 0x7f1d14071ab0 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.740+0000 7f1d0aa00640 1 -- 172.21.15.40:0/2181326891 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d000074c0 con 0x7f1d14071ab0 2024-03-08T06:04:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.740+0000 7f1d19a00640 1 -- 172.21.15.40:0/2181326891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1412de20 con 0x7f1d14071ab0 2024-03-08T06:04:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.742+0000 7f1d19a00640 1 -- 172.21.15.40:0/2181326891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d140cf330 con 0x7f1d14071ab0 2024-03-08T06:04:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.742+0000 7f1d0aa00640 1 -- 172.21.15.40:0/2181326891 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d0001b070 con 0x7f1d14071ab0 2024-03-08T06:04:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.743+0000 7f1d0aa00640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1cf406b4d0 0x7f1cf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.743+0000 7f1d0aa00640 1 -- 172.21.15.40:0/2181326891 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1d000895c0 con 0x7f1d14071ab0 2024-03-08T06:04:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.744+0000 7f1d12a00640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1cf406b4d0 0x7f1cf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.744+0000 7f1d12a00640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1cf406b4d0 0x7f1cf406d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d04036000 tx=0x7f1d04008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.748+0000 7f1d0aa00640 1 -- 172.21.15.40:0/2181326891 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d1415d0c0 con 0x7f1d14071ab0 2024-03-08T06:04:58.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.947+0000 7f1d19a00640 1 -- 172.21.15.40:0/2181326891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d14067cb0 con 0x7f1d14071ab0 2024-03-08T06:04:58.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.950+0000 7f1d0aa00640 1 -- 172.21.15.40:0/2181326891 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16770 (secure 0 0 0) 0x7f1d00058320 con 0x7f1d14071ab0 2024-03-08T06:04:58.950 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:04:58.951 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:04:48.785806+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:03:59.826823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T06:04:01.016829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:04:58.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 -- 172.21.15.40:0/2181326891 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1cf406b4d0 msgr2=0x7f1cf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1cf406b4d0 0x7f1cf406d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d04036000 tx=0x7f1d04008030 comp rx=0 tx=0).stop 2024-03-08T06:04:58.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 -- 172.21.15.40:0/2181326891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d14071ab0 msgr2=0x7f1d141e5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:58.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d14071ab0 0x7f1d141e5530 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000c7b0 tx=0x7f1d0000cc70 comp rx=0 tx=0).stop 2024-03-08T06:04:58.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 -- 172.21.15.40:0/2181326891 shutdown_connections 2024-03-08T06:04:58.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1cf406b4d0 0x7f1cf406d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1d14071e90 0x7f1d140cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d14071ab0 0x7f1d141e5530 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 --2- 172.21.15.40:0/2181326891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1406d600 0x7f1d1415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:58.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 -- 172.21.15.40:0/2181326891 >> 172.21.15.40:0/2181326891 conn(0x7f1d1406ae30 msgr2=0x7f1d140af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:58.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 -- 172.21.15.40:0/2181326891 shutdown_connections 2024-03-08T06:04:58.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:58.953+0000 7f1d19a00640 1 -- 172.21.15.40:0/2181326891 wait complete. 2024-03-08T06:04:58.973 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-08T06:04:59.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.190+0000 7fad04c00640 1 Processor -- start 2024-03-08T06:04:59.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.190+0000 7fad04c00640 1 -- start start 2024-03-08T06:04:59.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.191+0000 7fad04c00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad001656e0 0x7fad001666c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.191+0000 7fad04c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000ce130 0x7fad000ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.191+0000 7fad04c00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad00166c00 con 0x7fad000d0810 2024-03-08T06:04:59.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.191+0000 7fad04c00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad00165ac0 con 0x7fad001656e0 2024-03-08T06:04:59.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.191+0000 7fad04c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad000d2a30 con 0x7fad000ce130 2024-03-08T06:04:59.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.192+0000 7facfea00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fad000d0810 0x7fad000d2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48122/0 (socket says 172.21.15.40:48122) 2024-03-08T06:04:59.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.192+0000 7facfea00640 1 -- 172.21.15.40:0/218147547 learned_addr learned my addr 172.21.15.40:0/218147547 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.192+0000 7facff400640 1 --2- 172.21.15.40:0/218147547 >> v2:172.21.15.53:3300/0 conn(0x7fad001656e0 0x7fad001666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.192+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638251365 0 0) 0x7fad00166c00 con 0x7fad000d0810 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.192+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face0003520 con 0x7fad000d0810 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.192+0000 7facffe00640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000ce130 0x7fad000ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2989521458 0 0) 0x7face0003520 con 0x7fad000d0810 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000ce130 msgr2=0x7fad000ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000ce130 0x7fad000ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 >> v2:172.21.15.53:3300/0 conn(0x7fad001656e0 msgr2=0x7fad001666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 --2- 172.21.15.40:0/218147547 >> v2:172.21.15.53:3300/0 conn(0x7fad001656e0 0x7fad001666c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad000d0d80 con 0x7fad000d0810 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7face80019f0 con 0x7fad000d0810 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 >> v1:172.21.15.40:6789/0 conn(0x7fad000d0810 legacy=0x7fad000d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 0x7face0003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000ce130 0x7face0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face0004db0 0x7face0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad00165ac0 con 0x7face0003a90 2024-03-08T06:04:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000d2a30 con 0x7face0004db0 2024-03-08T06:04:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face000bce0 con 0x7fad000ce130 2024-03-08T06:04:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facff400640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 0x7face0003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facff400640 1 -- 172.21.15.40:0/218147547 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000ce130 msgr2=0x7face0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.193+0000 7facfea00640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000ce130 0x7face0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.194+0000 7facff400640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000ce130 0x7face0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.194+0000 7facff400640 1 -- 172.21.15.40:0/218147547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face0004db0 msgr2=0x7face0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.194+0000 7facffe00640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face0004db0 0x7face0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.194+0000 7facff400640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face0004db0 0x7face0005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.194+0000 7facff400640 1 -- 172.21.15.40:0/218147547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face0003520 con 0x7face0003a90 2024-03-08T06:04:59.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.195+0000 7facff400640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 0x7face0003eb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7facf000ba30 tx=0x7face0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5288fd5bf572ef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:59.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.195+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf000b600 con 0x7face0003a90 2024-03-08T06:04:59.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.195+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf0038070 con 0x7face0003a90 2024-03-08T06:04:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.195+0000 7facfe000640 1 -- 172.21.15.40:0/218147547 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf002d040 con 0x7face0003a90 2024-03-08T06:04:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.196+0000 7fad04c00640 1 -- 172.21.15.40:0/218147547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 msgr2=0x7face0003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.196+0000 7fad04c00640 1 --2- 172.21.15.40:0/218147547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 0x7face0003eb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7facf000ba30 tx=0x7face0005830 comp rx=0 tx=0).stop 2024-03-08T06:04:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.196+0000 7facffe00640 1 -- 172.21.15.40:0/218147547 reap_dead start 2024-03-08T06:04:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.196+0000 7fad04c00640 1 -- 172.21.15.40:0/218147547 shutdown_connections 2024-03-08T06:04:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.196+0000 7fad04c00640 1 -- 172.21.15.40:0/218147547 >> 172.21.15.40:0/218147547 conn(0x7fad000b59e0 msgr2=0x7fad000b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.196+0000 7fad04c00640 1 -- 172.21.15.40:0/218147547 shutdown_connections 2024-03-08T06:04:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.196+0000 7fad04c00640 1 -- 172.21.15.40:0/218147547 wait complete. 2024-03-08T06:04:59.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.198+0000 7fad04c00640 1 Processor -- start 2024-03-08T06:04:59.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.198+0000 7fad04c00640 1 -- start start 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.199+0000 7fad04c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 0x7fad001528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.199+0000 7fad04c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000c8660 0x7fad000c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.199+0000 7fad04c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad0006e260 0x7fad0006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.199+0000 7fad04c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000d1b20 con 0x7face0003a90 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.199+0000 7fad04c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000d18d0 con 0x7fad0006e260 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.199+0000 7fad04c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad000ce5c0 con 0x7fad000c8660 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.199+0000 7facff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 0x7fad001528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.199+0000 7facff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 0x7fad001528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53390/0 (socket says 172.21.15.40:53390) 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.199+0000 7facff400640 1 -- 172.21.15.40:0/3729198252 learned_addr learned my addr 172.21.15.40:0/3729198252 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.200+0000 7facfea00640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000c8660 0x7fad000c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.200+0000 7facffe00640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad0006e260 0x7fad0006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.200+0000 7facff400640 1 -- 172.21.15.40:0/3729198252 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000c8660 msgr2=0x7fad000c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.200+0000 7facff400640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000c8660 0x7fad000c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.200+0000 7facff400640 1 -- 172.21.15.40:0/3729198252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad0006e260 msgr2=0x7fad0006e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.200+0000 7facff400640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad0006e260 0x7fad0006e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.200+0000 7facff400640 1 -- 172.21.15.40:0/3729198252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0006efd0 con 0x7face0003a90 2024-03-08T06:04:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.200+0000 7facffe00640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad0006e260 0x7fad0006e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:04:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.200+0000 7facff400640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 0x7fad001528f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7facf0035040 tx=0x7facf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.200+0000 7facf7400640 1 -- 172.21.15.40:0/3729198252 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0042070 con 0x7face0003a90 2024-03-08T06:04:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.201+0000 7facf7400640 1 -- 172.21.15.40:0/3729198252 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf0033e20 con 0x7face0003a90 2024-03-08T06:04:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.201+0000 7facf7400640 1 -- 172.21.15.40:0/3729198252 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0004260 con 0x7face0003a90 2024-03-08T06:04:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.201+0000 7fad04c00640 1 -- 172.21.15.40:0/3729198252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad00153150 con 0x7face0003a90 2024-03-08T06:04:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.201+0000 7fad04c00640 1 -- 172.21.15.40:0/3729198252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad00153710 con 0x7face0003a90 2024-03-08T06:04:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.202+0000 7fad04c00640 1 -- 172.21.15.40:0/3729198252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad00075090 con 0x7face0003a90 2024-03-08T06:04:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.203+0000 7facf7400640 1 -- 172.21.15.40:0/3729198252 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7facf0003610 con 0x7face0003a90 2024-03-08T06:04:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.204+0000 7facf7400640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7facdc06b500 0x7facdc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.204+0000 7facf7400640 1 -- 172.21.15.40:0/3729198252 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7facf00abfe0 con 0x7face0003a90 2024-03-08T06:04:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.204+0000 7facfea00640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7facdc06b500 0x7facdc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.205+0000 7facfea00640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7facdc06b500 0x7facdc06d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7face8001370 tx=0x7face8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:59.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.209+0000 7facf7400640 1 -- 172.21.15.40:0/3729198252 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7facf007ad60 con 0x7face0003a90 2024-03-08T06:04:59.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.449+0000 7fad04c00640 1 -- 172.21.15.40:0/3729198252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad000d16b0 con 0x7face0003a90 2024-03-08T06:04:59.450 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-03-08T06:04:59.450+0000 7fce8d000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-08T06:04:59.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.453+0000 7facf7400640 1 -- 172.21.15.40:0/3729198252 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7facf007a660 con 0x7face0003a90 2024-03-08T06:04:59.453 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-08T06:04:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.456+0000 7fad04c00640 1 -- 172.21.15.40:0/3729198252 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7facdc06b500 msgr2=0x7facdc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.456+0000 7fad04c00640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7facdc06b500 0x7facdc06d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7face8001370 tx=0x7face8009040 comp rx=0 tx=0).stop 2024-03-08T06:04:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.456+0000 7fad04c00640 1 -- 172.21.15.40:0/3729198252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 msgr2=0x7fad001528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.456+0000 7fad04c00640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 0x7fad001528f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7facf0035040 tx=0x7facf0004730 comp rx=0 tx=0).stop 2024-03-08T06:04:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.457+0000 7fad04c00640 1 -- 172.21.15.40:0/3729198252 shutdown_connections 2024-03-08T06:04:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.457+0000 7fad04c00640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7facdc06b500 0x7facdc06d9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.457+0000 7fad04c00640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad0006e260 0x7fad0006e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.457+0000 7fad04c00640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fad000c8660 0x7fad000c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.457+0000 7fad04c00640 1 --2- 172.21.15.40:0/3729198252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face0003a90 0x7fad001528f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.457+0000 7fad04c00640 1 -- 172.21.15.40:0/3729198252 >> 172.21.15.40:0/3729198252 conn(0x7fad000b59e0 msgr2=0x7fad000d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:59.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.457+0000 7fad04c00640 1 -- 172.21.15.40:0/3729198252 shutdown_connections 2024-03-08T06:04:59.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.457+0000 7fad04c00640 1 -- 172.21.15.40:0/3729198252 wait complete. 2024-03-08T06:04:59.478 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-08T06:04:59.497 INFO:tasks.ceph.mds.a.smithi040.stderr: -7837> 2024-03-08T06:04:04.715+0000 7f3af9000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-08T06:04:59.577 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-03-08T06:04:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.698+0000 7fe81c600640 1 Processor -- start 2024-03-08T06:04:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.698+0000 7fe81c600640 1 -- start start 2024-03-08T06:04:59.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.700+0000 7fe81c600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe814144710 0x7fe8140cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.700+0000 7fe81c600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81415ee70 con 0x7fe814071bc0 2024-03-08T06:04:59.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.700+0000 7fe81c600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81415f150 con 0x7fe814071fa0 2024-03-08T06:04:59.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.700+0000 7fe81c600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81415f430 con 0x7fe814144710 2024-03-08T06:04:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.700+0000 7fe819c00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe814071bc0 0x7fe81415b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48136/0 (socket says 172.21.15.40:48136) 2024-03-08T06:04:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.700+0000 7fe819c00640 1 -- 172.21.15.40:0/3245247734 learned_addr learned my addr 172.21.15.40:0/3245247734 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:04:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.700+0000 7fe81a600640 1 --2- 172.21.15.40:0/3245247734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe814144710 0x7fe8140cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544955115 0 0) 0x7fe81415ee70 con 0x7fe814071bc0 2024-03-08T06:04:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7fc003520 con 0x7fe814071bc0 2024-03-08T06:04:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 400088526 0 0) 0x7fe7fc003520 con 0x7fe814071bc0 2024-03-08T06:04:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe814144710 msgr2=0x7fe8140cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 --2- 172.21.15.40:0/3245247734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe814144710 0x7fe8140cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 >> v1:172.21.15.53:6789/0 conn(0x7fe814071fa0 legacy=0x7fe814144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81415e760 con 0x7fe814071bc0 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fe808001670 con 0x7fe814071bc0 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 >> v1:172.21.15.40:6789/0 conn(0x7fe814071bc0 legacy=0x7fe81415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 --2- 172.21.15.40:0/3245247734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7fc0039e0 0x7fe7fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 --2- 172.21.15.40:0/3245247734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc004800 0x7fe7fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 --2- 172.21.15.40:0/3245247734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe814144710 0x7fe7fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81415f430 con 0x7fe7fc0039e0 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc00dcd0 con 0x7fe7fc004800 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.701+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc00ded0 con 0x7fe814144710 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.702+0000 7fe819c00640 1 --2- 172.21.15.40:0/3245247734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7fc0039e0 0x7fe7fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.702+0000 7fe819200640 1 --2- 172.21.15.40:0/3245247734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc004800 0x7fe7fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.702+0000 7fe819c00640 1 -- 172.21.15.40:0/3245247734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe814144710 msgr2=0x7fe7fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.702+0000 7fe819c00640 1 --2- 172.21.15.40:0/3245247734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe814144710 0x7fe7fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.702+0000 7fe819c00640 1 -- 172.21.15.40:0/3245247734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc004800 msgr2=0x7fe7fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:04:59.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.702+0000 7fe819c00640 1 --2- 172.21.15.40:0/3245247734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc004800 0x7fe7fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.702+0000 7fe819c00640 1 -- 172.21.15.40:0/3245247734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc003520 con 0x7fe7fc0039e0 2024-03-08T06:04:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.703+0000 7fe819c00640 1 --2- 172.21.15.40:0/3245247734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7fc0039e0 0x7fe7fc003dc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe80800c160 tx=0x7fe7fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f235d51d935f09 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.704+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808007c80 con 0x7fe7fc0039e0 2024-03-08T06:04:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.704+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe808030070 con 0x7fe7fc0039e0 2024-03-08T06:04:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.704+0000 7fe813e00640 1 -- 172.21.15.40:0/3245247734 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808005d30 con 0x7fe7fc0039e0 2024-03-08T06:04:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.704+0000 7fe81c600640 1 -- 172.21.15.40:0/3245247734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7fc0039e0 msgr2=0x7fe7fc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.704+0000 7fe81c600640 1 --2- 172.21.15.40:0/3245247734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7fc0039e0 0x7fe7fc003dc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe80800c160 tx=0x7fe7fc005830 comp rx=0 tx=0).stop 2024-03-08T06:04:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.704+0000 7fe81a600640 1 -- 172.21.15.40:0/3245247734 reap_dead start 2024-03-08T06:04:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.704+0000 7fe81c600640 1 -- 172.21.15.40:0/3245247734 shutdown_connections 2024-03-08T06:04:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.704+0000 7fe81c600640 1 -- 172.21.15.40:0/3245247734 >> 172.21.15.40:0/3245247734 conn(0x7fe81406ae30 msgr2=0x7fe81406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:04:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.704+0000 7fe81c600640 1 -- 172.21.15.40:0/3245247734 shutdown_connections 2024-03-08T06:04:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.704+0000 7fe81c600640 1 -- 172.21.15.40:0/3245247734 wait complete. 2024-03-08T06:04:59.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.706+0000 7fe81c600640 1 Processor -- start 2024-03-08T06:04:59.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.706+0000 7fe81c600640 1 -- start start 2024-03-08T06:04:59.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.707+0000 7fe81c600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc0039e0 0x7fe81415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.707+0000 7fe81c600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8140aeb40 0x7fe8140aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.707+0000 7fe81c600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8140ac8e0 0x7fe8140acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.707+0000 7fe81c600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8140cf5b0 con 0x7fe8140ac8e0 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.707+0000 7fe81c600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81415ec20 con 0x7fe7fc0039e0 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.707+0000 7fe81c600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8140cf730 con 0x7fe8140aeb40 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.708+0000 7fe81a600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8140ac8e0 0x7fe8140acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.708+0000 7fe81a600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8140ac8e0 0x7fe8140acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53422/0 (socket says 172.21.15.40:53422) 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.708+0000 7fe81a600640 1 -- 172.21.15.40:0/2082573014 learned_addr learned my addr 172.21.15.40:0/2082573014 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.708+0000 7fe819c00640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc0039e0 0x7fe81415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.708+0000 7fe819200640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8140aeb40 0x7fe8140aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.708+0000 7fe81a600640 1 -- 172.21.15.40:0/2082573014 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8140aeb40 msgr2=0x7fe8140aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.708+0000 7fe81a600640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8140aeb40 0x7fe8140aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.708+0000 7fe81a600640 1 -- 172.21.15.40:0/2082573014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc0039e0 msgr2=0x7fe81415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.708+0000 7fe81a600640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc0039e0 0x7fe81415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.708+0000 7fe81a600640 1 -- 172.21.15.40:0/2082573014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8140ad5d0 con 0x7fe8140ac8e0 2024-03-08T06:04:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.708+0000 7fe819c00640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc0039e0 0x7fe81415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:04:59.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.709+0000 7fe81a600640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8140ac8e0 0x7fe8140acd60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc004780 tx=0x7fe80c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:59.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.709+0000 7fe811600640 1 -- 172.21.15.40:0/2082573014 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c00bce0 con 0x7fe8140ac8e0 2024-03-08T06:04:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.709+0000 7fe811600640 1 -- 172.21.15.40:0/2082573014 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c01a070 con 0x7fe8140ac8e0 2024-03-08T06:04:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.709+0000 7fe811600640 1 -- 172.21.15.40:0/2082573014 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c014640 con 0x7fe8140ac8e0 2024-03-08T06:04:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.709+0000 7fe81c600640 1 -- 172.21.15.40:0/2082573014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8140ad8b0 con 0x7fe8140ac8e0 2024-03-08T06:04:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.709+0000 7fe81c600640 1 -- 172.21.15.40:0/2082573014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8140ade70 con 0x7fe8140ac8e0 2024-03-08T06:04:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.711+0000 7fe81c600640 1 -- 172.21.15.40:0/2082573014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe814067cb0 con 0x7fe8140ac8e0 2024-03-08T06:04:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.711+0000 7fe811600640 1 -- 172.21.15.40:0/2082573014 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe80c005d10 con 0x7fe8140ac8e0 2024-03-08T06:04:59.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.712+0000 7fe811600640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe7f406b500 0x7fe7f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:04:59.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.712+0000 7fe811600640 1 -- 172.21.15.40:0/2082573014 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fe80c08a3d0 con 0x7fe8140ac8e0 2024-03-08T06:04:59.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.713+0000 7fe819c00640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe7f406b500 0x7fe7f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:04:59.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.713+0000 7fe819c00640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe7f406b500 0x7fe7f406d9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe808034000 tx=0x7fe808009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:04:59.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.717+0000 7fe811600640 1 -- 172.21.15.40:0/2082573014 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe80c0590f0 con 0x7fe8140ac8e0 2024-03-08T06:04:59.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:04:59.954+0000 7fe81c600640 1 -- 172.21.15.40:0/2082573014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe8140b2d90 con 0x7fe8140ac8e0 2024-03-08T06:05:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.463+0000 7fe811600640 1 -- 172.21.15.40:0/2082573014 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe80c0589f0 con 0x7fe8140ac8e0 2024-03-08T06:05:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.467+0000 7fe81c600640 1 -- 172.21.15.40:0/2082573014 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe7f406b500 msgr2=0x7fe7f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.467+0000 7fe81c600640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe7f406b500 0x7fe7f406d9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe808034000 tx=0x7fe808009030 comp rx=0 tx=0).stop 2024-03-08T06:05:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.467+0000 7fe81c600640 1 -- 172.21.15.40:0/2082573014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8140ac8e0 msgr2=0x7fe8140acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.467+0000 7fe81c600640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8140ac8e0 0x7fe8140acd60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc004780 tx=0x7fe80c00b480 comp rx=0 tx=0).stop 2024-03-08T06:05:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.468+0000 7fe81c600640 1 -- 172.21.15.40:0/2082573014 shutdown_connections 2024-03-08T06:05:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.468+0000 7fe81c600640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe7f406b500 0x7fe7f406d9c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.468+0000 7fe81c600640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8140ac8e0 0x7fe8140acd60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.468+0000 7fe81c600640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8140aeb40 0x7fe8140aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.468+0000 7fe81c600640 1 --2- 172.21.15.40:0/2082573014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc0039e0 0x7fe81415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.468+0000 7fe81c600640 1 -- 172.21.15.40:0/2082573014 >> 172.21.15.40:0/2082573014 conn(0x7fe81406ae30 msgr2=0x7fe81406ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.468+0000 7fe81c600640 1 -- 172.21.15.40:0/2082573014 shutdown_connections 2024-03-08T06:05:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.468+0000 7fe81c600640 1 -- 172.21.15.40:0/2082573014 wait complete. 2024-03-08T06:05:00.488 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-08T06:05:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.760+0000 7f206b000640 1 Processor -- start 2024-03-08T06:05:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.760+0000 7f206b000640 1 -- start start 2024-03-08T06:05:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.762+0000 7f206b000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2064071b40 0x7f20640cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.762+0000 7f206b000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206415c9e0 con 0x7f206415bf90 2024-03-08T06:05:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.762+0000 7f206b000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206415ccc0 con 0x7f2064145200 2024-03-08T06:05:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.762+0000 7f206b000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206406e3f0 con 0x7f2064071b40 2024-03-08T06:05:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.762+0000 7f2069200640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f206415bf90 0x7f2064071430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48140/0 (socket says 172.21.15.40:48140) 2024-03-08T06:05:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.762+0000 7f2069200640 1 -- 172.21.15.40:0/4206720453 learned_addr learned my addr 172.21.15.40:0/4206720453 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f206a600640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2064071b40 0x7f20640cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520960277 0 0) 0x7f206415c9e0 con 0x7f206415bf90 2024-03-08T06:05:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f204c003520 con 0x7f206415bf90 2024-03-08T06:05:00.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2921879070 0 0) 0x7f204c003520 con 0x7f206415bf90 2024-03-08T06:05:00.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2064071b40 msgr2=0x7f20640cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:00.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2064071b40 0x7f20640cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:00.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 >> v1:172.21.15.53:6789/0 conn(0x7f2064145200 legacy=0x7f206406dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:00.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206406d4e0 con 0x7f206415bf90 2024-03-08T06:05:00.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f2050001670 con 0x7f206415bf90 2024-03-08T06:05:00.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 >> v1:172.21.15.40:6789/0 conn(0x7f206415bf90 legacy=0x7f2064071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:00.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2064071b40 0x7f204c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:00.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c003f60 0x7f204c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:00.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f204c004d40 0x7f204c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:00.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206406e3f0 con 0x7f204c003f60 2024-03-08T06:05:00.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c00baa0 con 0x7f204c004d40 2024-03-08T06:05:00.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.763+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c00bca0 con 0x7f2064071b40 2024-03-08T06:05:00.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.764+0000 7f2069200640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c003f60 0x7f204c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:00.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.764+0000 7f2069c00640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2064071b40 0x7f204c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:00.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.764+0000 7f2069200640 1 -- 172.21.15.40:0/4206720453 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2064071b40 msgr2=0x7f204c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:00.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.764+0000 7f2069200640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2064071b40 0x7f204c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:00.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.764+0000 7f206a600640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f204c004d40 0x7f204c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:00.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.764+0000 7f2069200640 1 -- 172.21.15.40:0/4206720453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f204c004d40 msgr2=0x7f204c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:00.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.764+0000 7f2069200640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f204c004d40 0x7f204c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:00.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.764+0000 7f2069200640 1 -- 172.21.15.40:0/4206720453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c003520 con 0x7f204c003f60 2024-03-08T06:05:00.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.765+0000 7f2069200640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c003f60 0x7f204c004800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f205000c240 tx=0x7f204c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bc3eaad47dad67 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:00.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.766+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050007d10 con 0x7f204c003f60 2024-03-08T06:05:00.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.766+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2050030070 con 0x7f204c003f60 2024-03-08T06:05:00.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.766+0000 7f2063e00640 1 -- 172.21.15.40:0/4206720453 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205002cda0 con 0x7f204c003f60 2024-03-08T06:05:00.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.766+0000 7f206b000640 1 -- 172.21.15.40:0/4206720453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c003f60 msgr2=0x7f204c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:00.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.766+0000 7f206b000640 1 --2- 172.21.15.40:0/4206720453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c003f60 0x7f204c004800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f205000c240 tx=0x7f204c005830 comp rx=0 tx=0).stop 2024-03-08T06:05:00.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.766+0000 7f206a600640 1 -- 172.21.15.40:0/4206720453 reap_dead start 2024-03-08T06:05:00.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.766+0000 7f206b000640 1 -- 172.21.15.40:0/4206720453 shutdown_connections 2024-03-08T06:05:00.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.766+0000 7f206b000640 1 -- 172.21.15.40:0/4206720453 >> 172.21.15.40:0/4206720453 conn(0x7f20640b1600 msgr2=0x7f20640b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:00.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.766+0000 7f206b000640 1 -- 172.21.15.40:0/4206720453 shutdown_connections 2024-03-08T06:05:00.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.766+0000 7f206b000640 1 -- 172.21.15.40:0/4206720453 wait complete. 2024-03-08T06:05:00.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.768+0000 7f206b000640 1 Processor -- start 2024-03-08T06:05:00.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.768+0000 7f206b000640 1 -- start start 2024-03-08T06:05:00.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.769+0000 7f206b000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f204c003f60 0x7f206415b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:00.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.769+0000 7f206b000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20641e7ff0 0x7f20641e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:00.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.769+0000 7f206b000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20641e92a0 0x7f20641339a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:00.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.769+0000 7f206b000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20641455e0 con 0x7f20641e92a0 2024-03-08T06:05:00.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.769+0000 7f206b000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206406e9a0 con 0x7f20641e7ff0 2024-03-08T06:05:00.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.769+0000 7f206b000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206415c420 con 0x7f204c003f60 2024-03-08T06:05:00.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.769+0000 7f206a600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20641e92a0 0x7f20641339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:00.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.769+0000 7f206a600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20641e92a0 0x7f20641339a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53448/0 (socket says 172.21.15.40:53448) 2024-03-08T06:05:00.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.770+0000 7f206a600640 1 -- 172.21.15.40:0/4261174233 learned_addr learned my addr 172.21.15.40:0/4261174233 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:00.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.770+0000 7f2069200640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20641e7ff0 0x7f20641e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:00.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.770+0000 7f2069c00640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f204c003f60 0x7f206415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:00.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.770+0000 7f206a600640 1 -- 172.21.15.40:0/4261174233 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f204c003f60 msgr2=0x7f206415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:00.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.770+0000 7f206a600640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f204c003f60 0x7f206415b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:00.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.770+0000 7f206a600640 1 -- 172.21.15.40:0/4261174233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20641e7ff0 msgr2=0x7f20641e8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:00.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.770+0000 7f206a600640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20641e7ff0 0x7f20641e8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:00.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.770+0000 7f206a600640 1 -- 172.21.15.40:0/4261174233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206415b770 con 0x7f20641e92a0 2024-03-08T06:05:00.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.770+0000 7f206a600640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20641e92a0 0x7f20641339a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2064072630 tx=0x7f205c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:00.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.771+0000 7f2061600640 1 -- 172.21.15.40:0/4261174233 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c01c070 con 0x7f20641e92a0 2024-03-08T06:05:00.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.771+0000 7f2061600640 1 -- 172.21.15.40:0/4261174233 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205c017400 con 0x7f20641e92a0 2024-03-08T06:05:00.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.771+0000 7f2061600640 1 -- 172.21.15.40:0/4261174233 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c0177f0 con 0x7f20641e92a0 2024-03-08T06:05:00.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.771+0000 7f206b000640 1 -- 172.21.15.40:0/4261174233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20641e9740 con 0x7f20641e92a0 2024-03-08T06:05:00.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.771+0000 7f206b000640 1 -- 172.21.15.40:0/4261174233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20641e77f0 con 0x7f20641e92a0 2024-03-08T06:05:00.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.772+0000 7f206b000640 1 -- 172.21.15.40:0/4261174233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2064075100 con 0x7f20641e92a0 2024-03-08T06:05:00.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.773+0000 7f2061600640 1 -- 172.21.15.40:0/4261174233 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f205c005d10 con 0x7f20641e92a0 2024-03-08T06:05:00.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.774+0000 7f2061600640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f204406b4d0 0x7f204406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:00.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.774+0000 7f2061600640 1 -- 172.21.15.40:0/4261174233 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f205c08a7a0 con 0x7f20641e92a0 2024-03-08T06:05:00.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.774+0000 7f2069c00640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f204406b4d0 0x7f204406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:00.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.775+0000 7f2069c00640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f204406b4d0 0x7f204406d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2058007fd0 tx=0x7f2058009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:00.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.779+0000 7f2061600640 1 -- 172.21.15.40:0/4261174233 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f206415b770 con 0x7f20641e92a0 2024-03-08T06:05:00.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:00.968+0000 7f206b000640 1 -- 172.21.15.40:0/4261174233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f206406e780 con 0x7f20641e92a0 2024-03-08T06:05:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.878+0000 7f2061600640 1 -- 172.21.15.40:0/4261174233 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f205c012070 con 0x7f20641e92a0 2024-03-08T06:05:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-03-08T06:05:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.883+0000 7f206b000640 1 -- 172.21.15.40:0/4261174233 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f204406b4d0 msgr2=0x7f204406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.883+0000 7f206b000640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f204406b4d0 0x7f204406d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2058007fd0 tx=0x7f2058009040 comp rx=0 tx=0).stop 2024-03-08T06:05:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.883+0000 7f206b000640 1 -- 172.21.15.40:0/4261174233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20641e92a0 msgr2=0x7f20641339a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.883+0000 7f206b000640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20641e92a0 0x7f20641339a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2064072630 tx=0x7f205c00f4a0 comp rx=0 tx=0).stop 2024-03-08T06:05:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.884+0000 7f206b000640 1 -- 172.21.15.40:0/4261174233 shutdown_connections 2024-03-08T06:05:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.884+0000 7f206b000640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f204406b4d0 0x7f204406d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:01.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.884+0000 7f206b000640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20641e92a0 0x7f20641339a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.884+0000 7f206b000640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20641e7ff0 0x7f20641e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.884+0000 7f206b000640 1 --2- 172.21.15.40:0/4261174233 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f204c003f60 0x7f206415b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.884+0000 7f206b000640 1 -- 172.21.15.40:0/4261174233 >> 172.21.15.40:0/4261174233 conn(0x7f20640b1600 msgr2=0x7f2064071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.884+0000 7f206b000640 1 -- 172.21.15.40:0/4261174233 shutdown_connections 2024-03-08T06:05:01.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:01.884+0000 7f206b000640 1 -- 172.21.15.40:0/4261174233 wait complete. 2024-03-08T06:05:01.907 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-08T06:05:02.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.136+0000 7f8263200640 1 Processor -- start 2024-03-08T06:05:02.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.136+0000 7f8263200640 1 -- start start 2024-03-08T06:05:02.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.138+0000 7f8263200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826406d520 0x7f8264071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:02.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.138+0000 7f8263200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8264071a60 0x7f82640cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:02.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.138+0000 7f8263200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826406dac0 con 0x7f8264144770 2024-03-08T06:05:02.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.138+0000 7f8263200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826406dda0 con 0x7f8264071a60 2024-03-08T06:05:02.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.138+0000 7f8263200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826406e080 con 0x7f826406d520 2024-03-08T06:05:02.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.138+0000 7f8262800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8264071a60 0x7f82640cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:02.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.138+0000 7f8262800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8264071a60 0x7f82640cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53462/0 (socket says 172.21.15.40:53462) 2024-03-08T06:05:02.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.138+0000 7f8262800640 1 -- 172.21.15.40:0/4006273355 learned_addr learned my addr 172.21.15.40:0/4006273355 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:02.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.138+0000 7f8261400640 1 --2- 172.21.15.40:0/4006273355 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826406d520 0x7f8264071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:02.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.138+0000 7f8262800640 1 -- 172.21.15.40:0/4006273355 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826406d520 msgr2=0x7f8264071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:02.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.139+0000 7f8262800640 1 --2- 172.21.15.40:0/4006273355 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826406d520 0x7f8264071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:02.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.139+0000 7f8262800640 1 -- 172.21.15.40:0/4006273355 >> v1:172.21.15.53:6789/0 conn(0x7f8264144770 legacy=0x7f82641451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:02.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.139+0000 7f8262800640 1 -- 172.21.15.40:0/4006273355 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826406e360 con 0x7f8264071a60 2024-03-08T06:05:02.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.140+0000 7f8262800640 1 --2- 172.21.15.40:0/4006273355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8264071a60 0x7f82640cc7f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f825c009e50 tx=0x7f825c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a0cf43628edb87 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:02.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.140+0000 7f8260a00640 1 -- 172.21.15.40:0/4006273355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c02c9e0 con 0x7f8264071a60 2024-03-08T06:05:02.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.140+0000 7f8260a00640 1 -- 172.21.15.40:0/4006273355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c02cb80 con 0x7f8264071a60 2024-03-08T06:05:02.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.140+0000 7f8260a00640 1 -- 172.21.15.40:0/4006273355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c0343f0 con 0x7f8264071a60 2024-03-08T06:05:02.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.141+0000 7f8263200640 1 -- 172.21.15.40:0/4006273355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8264071a60 msgr2=0x7f82640cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:02.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.141+0000 7f8263200640 1 --2- 172.21.15.40:0/4006273355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8264071a60 0x7f82640cc7f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f825c009e50 tx=0x7f825c02be40 comp rx=0 tx=0).stop 2024-03-08T06:05:02.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.141+0000 7f8263200640 1 -- 172.21.15.40:0/4006273355 shutdown_connections 2024-03-08T06:05:02.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.141+0000 7f8263200640 1 --2- 172.21.15.40:0/4006273355 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826406d520 0x7f8264071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:02.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.141+0000 7f8263200640 1 --2- 172.21.15.40:0/4006273355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8264071a60 0x7f82640cc7f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:02.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.141+0000 7f8263200640 1 -- 172.21.15.40:0/4006273355 >> 172.21.15.40:0/4006273355 conn(0x7f826406ac50 msgr2=0x7f826406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:02.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.141+0000 7f8263200640 1 -- 172.21.15.40:0/4006273355 shutdown_connections 2024-03-08T06:05:02.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.141+0000 7f8263200640 1 -- 172.21.15.40:0/4006273355 wait complete. 2024-03-08T06:05:02.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.143+0000 7f8263200640 1 Processor -- start 2024-03-08T06:05:02.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.143+0000 7f8263200640 1 -- start start 2024-03-08T06:05:02.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.144+0000 7f8263200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f826406d520 0x7f826415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:02.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.144+0000 7f8263200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8264071a60 0x7f8264142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:02.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.144+0000 7f8263200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8264144770 0x7f826415c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.144+0000 7f8263200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640cceb0 con 0x7f8264071a60 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.144+0000 7f8263200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640ccd30 con 0x7f826406d520 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.144+0000 7f8263200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82640cd030 con 0x7f8264144770 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.144+0000 7f8261400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8264071a60 0x7f8264142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.144+0000 7f8261400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8264071a60 0x7f8264142c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53464/0 (socket says 172.21.15.40:53464) 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.144+0000 7f8261400640 1 -- 172.21.15.40:0/4120604421 learned_addr learned my addr 172.21.15.40:0/4120604421 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f8262800640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8264144770 0x7f826415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f8261e00640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f826406d520 0x7f826415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f8261400640 1 -- 172.21.15.40:0/4120604421 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8264144770 msgr2=0x7f826415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f8261400640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8264144770 0x7f826415c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f8261400640 1 -- 172.21.15.40:0/4120604421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f826406d520 msgr2=0x7f826415aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:02.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f8261400640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f826406d520 0x7f826415aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f8261400640 1 -- 172.21.15.40:0/4120604421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82641436d0 con 0x7f8264071a60 2024-03-08T06:05:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f8262800640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8264144770 0x7f826415c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f8261400640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8264071a60 0x7f8264142c50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f825000d830 tx=0x7f825000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f825a000640 1 -- 172.21.15.40:0/4120604421 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000b660 con 0x7f8264071a60 2024-03-08T06:05:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f825a000640 1 -- 172.21.15.40:0/4120604421 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8250019070 con 0x7f8264071a60 2024-03-08T06:05:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f825a000640 1 -- 172.21.15.40:0/4120604421 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000ba20 con 0x7f8264071a60 2024-03-08T06:05:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.145+0000 7f8263200640 1 -- 172.21.15.40:0/4120604421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826413e620 con 0x7f8264071a60 2024-03-08T06:05:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.146+0000 7f8263200640 1 -- 172.21.15.40:0/4120604421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826413ebe0 con 0x7f8264071a60 2024-03-08T06:05:02.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.147+0000 7f8263200640 1 -- 172.21.15.40:0/4120604421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8264075090 con 0x7f8264071a60 2024-03-08T06:05:02.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.148+0000 7f825a000640 1 -- 172.21.15.40:0/4120604421 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f825001e460 con 0x7f8264071a60 2024-03-08T06:05:02.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.148+0000 7f825a000640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824406b4d0 0x7f824406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:02.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.149+0000 7f825a000640 1 -- 172.21.15.40:0/4120604421 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f825008a7b0 con 0x7f8264071a60 2024-03-08T06:05:02.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.149+0000 7f8261e00640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824406b4d0 0x7f824406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:02.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.149+0000 7f8261e00640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824406b4d0 0x7f824406d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8254006fd0 tx=0x7f8254008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:02.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.154+0000 7f825a000640 1 -- 172.21.15.40:0/4120604421 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8250059700 con 0x7f8264071a60 2024-03-08T06:05:02.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.354+0000 7f8263200640 1 -- 172.21.15.40:0/4120604421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f826406ea80 con 0x7f8264071a60 2024-03-08T06:05:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.889+0000 7f825a000640 1 -- 172.21.15.40:0/4120604421 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8250059000 con 0x7f8264071a60 2024-03-08T06:05:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-03-08T06:05:02.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.894+0000 7f8263200640 1 -- 172.21.15.40:0/4120604421 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824406b4d0 msgr2=0x7f824406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:02.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.894+0000 7f8263200640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824406b4d0 0x7f824406d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8254006fd0 tx=0x7f8254008040 comp rx=0 tx=0).stop 2024-03-08T06:05:02.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.894+0000 7f8263200640 1 -- 172.21.15.40:0/4120604421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8264071a60 msgr2=0x7f8264142c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:02.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.894+0000 7f8263200640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8264071a60 0x7f8264142c50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f825000d830 tx=0x7f825000dcf0 comp rx=0 tx=0).stop 2024-03-08T06:05:02.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.895+0000 7f8263200640 1 -- 172.21.15.40:0/4120604421 shutdown_connections 2024-03-08T06:05:02.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.895+0000 7f8263200640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f824406b4d0 0x7f824406d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:02.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.895+0000 7f8263200640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8264144770 0x7f826415c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:02.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.895+0000 7f8263200640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8264071a60 0x7f8264142c50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:02.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.895+0000 7f8263200640 1 --2- 172.21.15.40:0/4120604421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f826406d520 0x7f826415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:02.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.895+0000 7f8263200640 1 -- 172.21.15.40:0/4120604421 >> 172.21.15.40:0/4120604421 conn(0x7f826406ac50 msgr2=0x7f82640b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:02.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.895+0000 7f8263200640 1 -- 172.21.15.40:0/4120604421 shutdown_connections 2024-03-08T06:05:02.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:02.895+0000 7f8263200640 1 -- 172.21.15.40:0/4120604421 wait complete. 2024-03-08T06:05:02.920 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-08T06:05:02.920 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-08T06:05:02.920 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-08T06:05:02.920 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:05:02.920 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-08T06:05:02.921 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-08T06:05:02.921 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-08T06:05:02.921 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:05:02.921 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-08T06:05:02.921 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-08T06:05:02.921 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-08T06:05:02.921 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:05:02.921 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-08T06:05:02.922 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-08T06:05:02.922 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-08T06:05:02.922 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:05:02.942 INFO:tasks.ceph.mds.a:Stopped 2024-03-08T06:05:02.942 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-08T06:05:02.945 INFO:tasks.ceph.mds.c:Stopped 2024-03-08T06:05:02.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-08T06:05:02.946 INFO:tasks.ceph.mds.d:Stopped 2024-03-08T06:05:02.947 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-08T06:05:02.948 INFO:tasks.ceph.mds.b:Stopped 2024-03-08T06:05:02.948 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-08T06:05:02.992 INFO:tasks.ceph.mds.a:Started 2024-03-08T06:05:02.993 DEBUG:teuthology.parallel:result is None 2024-03-08T06:05:02.995 INFO:tasks.ceph.mds.d:Started 2024-03-08T06:05:02.995 DEBUG:teuthology.parallel:result is None 2024-03-08T06:05:02.999 INFO:tasks.ceph.mds.c:Started 2024-03-08T06:05:02.999 DEBUG:teuthology.parallel:result is None 2024-03-08T06:05:03.001 INFO:tasks.ceph.mds.b:Started 2024-03-08T06:05:03.001 DEBUG:teuthology.parallel:result is None 2024-03-08T06:05:03.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-08T06:05:03.152 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-03-08T06:05:03.153 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-03-08T06:05:03.161 INFO:tasks.ceph.mds.b.smithi053.stdout:starting mds.b at 2024-03-08T06:05:03.163 INFO:tasks.ceph.mds.d.smithi053.stdout:starting mds.d at 2024-03-08T06:05:03.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.194+0000 7faa74c00640 1 Processor -- start 2024-03-08T06:05:03.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.194+0000 7faa74c00640 1 -- start start 2024-03-08T06:05:03.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.195+0000 7faa74c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa700b27a0 0x7faa700d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:03.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.196+0000 7faa74c00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa700d3410 con 0x7faa700b5bf0 2024-03-08T06:05:03.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.196+0000 7faa74c00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa700d36f0 con 0x7faa701611f0 2024-03-08T06:05:03.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.196+0000 7faa74c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa700ceac0 con 0x7faa700b27a0 2024-03-08T06:05:03.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.196+0000 7faa67e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faa700b5bf0 0x7faa700d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48164/0 (socket says 172.21.15.40:48164) 2024-03-08T06:05:03.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.196+0000 7faa67e00640 1 -- 172.21.15.40:0/4123980487 learned_addr learned my addr 172.21.15.40:0/4123980487 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:03.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.196+0000 7faa6fe00640 1 --2- 172.21.15.40:0/4123980487 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa700b27a0 0x7faa700d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:03.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.196+0000 7faa6ea00640 1 -- 172.21.15.40:0/4123980487 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966832104 0 0) 0x7faa700d36f0 con 0x7faa701611f0 2024-03-08T06:05:03.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.196+0000 7faa6ea00640 1 -- 172.21.15.40:0/4123980487 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa54003520 con 0x7faa701611f0 2024-03-08T06:05:03.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.196+0000 7faa6ea00640 1 -- 172.21.15.40:0/4123980487 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249375063 0 0) 0x7faa700d3410 con 0x7faa700b5bf0 2024-03-08T06:05:03.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.196+0000 7faa6ea00640 1 -- 172.21.15.40:0/4123980487 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa700d36f0 con 0x7faa700b5bf0 2024-03-08T06:05:03.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.197+0000 7faa6fe00640 1 -- 172.21.15.40:0/4123980487 >> v1:172.21.15.53:6789/0 conn(0x7faa701611f0 legacy=0x7faa700b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:03.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.197+0000 7faa6fe00640 1 -- 172.21.15.40:0/4123980487 >> v1:172.21.15.40:6789/0 conn(0x7faa700b5bf0 legacy=0x7faa700d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:03.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.197+0000 7faa6fe00640 1 -- 172.21.15.40:0/4123980487 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa700ced10 con 0x7faa700b27a0 2024-03-08T06:05:03.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.198+0000 7faa6fe00640 1 --2- 172.21.15.40:0/4123980487 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa700b27a0 0x7faa700d0ff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faa68009dc0 tx=0x7faa6802be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ab88449a5087874 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:03.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.198+0000 7faa6ea00640 1 -- 172.21.15.40:0/4123980487 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68039070 con 0x7faa700b27a0 2024-03-08T06:05:03.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.198+0000 7faa6ea00640 1 -- 172.21.15.40:0/4123980487 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6802cbb0 con 0x7faa700b27a0 2024-03-08T06:05:03.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.198+0000 7faa6ea00640 1 -- 172.21.15.40:0/4123980487 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68034440 con 0x7faa700b27a0 2024-03-08T06:05:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.199+0000 7faa74c00640 1 -- 172.21.15.40:0/4123980487 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa700b27a0 msgr2=0x7faa700d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.199+0000 7faa74c00640 1 --2- 172.21.15.40:0/4123980487 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa700b27a0 0x7faa700d0ff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faa68009dc0 tx=0x7faa6802be00 comp rx=0 tx=0).stop 2024-03-08T06:05:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.199+0000 7faa74c00640 1 -- 172.21.15.40:0/4123980487 shutdown_connections 2024-03-08T06:05:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.199+0000 7faa74c00640 1 --2- 172.21.15.40:0/4123980487 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa700b27a0 0x7faa700d0ff0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.199+0000 7faa74c00640 1 -- 172.21.15.40:0/4123980487 >> 172.21.15.40:0/4123980487 conn(0x7faa7006c700 msgr2=0x7faa7006cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.199+0000 7faa74c00640 1 -- 172.21.15.40:0/4123980487 shutdown_connections 2024-03-08T06:05:03.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.199+0000 7faa74c00640 1 -- 172.21.15.40:0/4123980487 wait complete. 2024-03-08T06:05:03.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.201+0000 7faa74c00640 1 Processor -- start 2024-03-08T06:05:03.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.201+0000 7faa74c00640 1 -- start start 2024-03-08T06:05:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.202+0000 7faa74c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa700b27a0 0x7faa7014f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.202+0000 7faa74c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa700b5bf0 0x7faa700c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.202+0000 7faa74c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa701611f0 0x7faa700b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.202+0000 7faa74c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa700d1e40 con 0x7faa700b5bf0 2024-03-08T06:05:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.202+0000 7faa74c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa700cf080 con 0x7faa700b27a0 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.202+0000 7faa74c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa700d1fc0 con 0x7faa701611f0 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.202+0000 7faa67e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa700b5bf0 0x7faa700c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.202+0000 7faa67e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa700b5bf0 0x7faa700c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53504/0 (socket says 172.21.15.40:53504) 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.202+0000 7faa67e00640 1 -- 172.21.15.40:0/3159617568 learned_addr learned my addr 172.21.15.40:0/3159617568 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.202+0000 7faa6fe00640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa701611f0 0x7faa700b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.202+0000 7faa6f400640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa700b27a0 0x7faa7014f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa67e00640 1 -- 172.21.15.40:0/3159617568 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa701611f0 msgr2=0x7faa700b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa67e00640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa701611f0 0x7faa700b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa67e00640 1 -- 172.21.15.40:0/3159617568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa700b27a0 msgr2=0x7faa7014f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa67e00640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa700b27a0 0x7faa7014f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa67e00640 1 -- 172.21.15.40:0/3159617568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa701f3fc0 con 0x7faa700b5bf0 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa6f400640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa700b27a0 0x7faa7014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa67e00640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa700b5bf0 0x7faa700c93d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faa5800c430 tx=0x7faa5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa67400640 1 -- 172.21.15.40:0/3159617568 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58017070 con 0x7faa700b5bf0 2024-03-08T06:05:03.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa67400640 1 -- 172.21.15.40:0/3159617568 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58005ce0 con 0x7faa700b5bf0 2024-03-08T06:05:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa74c00640 1 -- 172.21.15.40:0/3159617568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa701f4ff0 con 0x7faa700b5bf0 2024-03-08T06:05:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa67400640 1 -- 172.21.15.40:0/3159617568 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa580072f0 con 0x7faa700b5bf0 2024-03-08T06:05:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.203+0000 7faa74c00640 1 -- 172.21.15.40:0/3159617568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa701f4510 con 0x7faa700b5bf0 2024-03-08T06:05:03.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.205+0000 7faa74c00640 1 -- 172.21.15.40:0/3159617568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa70075090 con 0x7faa700b5bf0 2024-03-08T06:05:03.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.206+0000 7faa67400640 1 -- 172.21.15.40:0/3159617568 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa5801a070 con 0x7faa700b5bf0 2024-03-08T06:05:03.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.207+0000 7faa67400640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faa4c06b4d0 0x7faa4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:03.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.208+0000 7faa67400640 1 -- 172.21.15.40:0/3159617568 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faa580893e0 con 0x7faa700b5bf0 2024-03-08T06:05:03.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.208+0000 7faa6f400640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faa4c06b4d0 0x7faa4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:03.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.210+0000 7faa6f400640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faa4c06b4d0 0x7faa4c06d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faa600011e0 tx=0x7faa60009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.215+0000 7faa67400640 1 -- 172.21.15.40:0/3159617568 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa58058560 con 0x7faa700b5bf0 2024-03-08T06:05:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.428+0000 7faa74c00640 1 -- 172.21.15.40:0/3159617568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7faa700b98b0 con 0x7faa700b5bf0 2024-03-08T06:05:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.912+0000 7faa67400640 1 -- 172.21.15.40:0/3159617568 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7faa58057e60 con 0x7faa700b5bf0 2024-03-08T06:05:03.913 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-03-08T06:05:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.916+0000 7faa74c00640 1 -- 172.21.15.40:0/3159617568 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faa4c06b4d0 msgr2=0x7faa4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.916+0000 7faa74c00640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faa4c06b4d0 0x7faa4c06d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faa600011e0 tx=0x7faa60009040 comp rx=0 tx=0).stop 2024-03-08T06:05:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.916+0000 7faa74c00640 1 -- 172.21.15.40:0/3159617568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa700b5bf0 msgr2=0x7faa700c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:03.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.916+0000 7faa74c00640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa700b5bf0 0x7faa700c93d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faa5800c430 tx=0x7faa5800c8f0 comp rx=0 tx=0).stop 2024-03-08T06:05:03.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.916+0000 7faa74c00640 1 -- 172.21.15.40:0/3159617568 shutdown_connections 2024-03-08T06:05:03.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.916+0000 7faa74c00640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7faa4c06b4d0 0x7faa4c06d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:03.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.916+0000 7faa74c00640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faa701611f0 0x7faa700b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:03.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.916+0000 7faa74c00640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa700b5bf0 0x7faa700c93d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:03.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.916+0000 7faa74c00640 1 --2- 172.21.15.40:0/3159617568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa700b27a0 0x7faa7014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:03.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.917+0000 7faa74c00640 1 -- 172.21.15.40:0/3159617568 >> 172.21.15.40:0/3159617568 conn(0x7faa7006c700 msgr2=0x7faa700d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:03.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.917+0000 7faa74c00640 1 -- 172.21.15.40:0/3159617568 shutdown_connections 2024-03-08T06:05:03.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:03.917+0000 7faa74c00640 1 -- 172.21.15.40:0/3159617568 wait complete. 2024-03-08T06:05:03.938 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-08T06:05:04.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.152+0000 7f98d3a00640 1 Processor -- start 2024-03-08T06:05:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.153+0000 7f98d3a00640 1 -- start start 2024-03-08T06:05:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.154+0000 7f98d3a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f98cc06db70 0x7f98cc06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.154+0000 7f98d3a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc09aca0 0x7f98cc0702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.154+0000 7f98d3a00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc09b210 con 0x7f98cc0707e0 2024-03-08T06:05:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.154+0000 7f98d3a00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc0cdee0 con 0x7f98cc06db70 2024-03-08T06:05:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.154+0000 7f98d3a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0cc110 con 0x7f98cc09aca0 2024-03-08T06:05:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.155+0000 7f98d1a00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f98cc0707e0 0x7f98cc0cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48176/0 (socket says 172.21.15.40:48176) 2024-03-08T06:05:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.155+0000 7f98d1a00640 1 -- 172.21.15.40:0/3363456003 learned_addr learned my addr 172.21.15.40:0/3363456003 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.155+0000 7f98d1000640 1 --2- 172.21.15.40:0/3363456003 >> v2:172.21.15.53:3300/0 conn(0x7f98cc06db70 0x7f98cc06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.155+0000 7f98cbe00640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc09aca0 0x7f98cc0702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.155+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326871883 0 0) 0x7f98cc09b210 con 0x7f98cc0707e0 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.155+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98b0003520 con 0x7f98cc0707e0 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.155+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1626575987 0 0) 0x7f98b0003520 con 0x7f98cc0707e0 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.155+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc09aca0 msgr2=0x7f98cc0702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.155+0000 7f98cb400640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc09aca0 0x7f98cc0702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 >> v2:172.21.15.53:3300/0 conn(0x7f98cc06db70 msgr2=0x7f98cc06faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cb400640 1 --2- 172.21.15.40:0/3363456003 >> v2:172.21.15.53:3300/0 conn(0x7f98cc06db70 0x7f98cc06faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0cc3f0 con 0x7f98cc0707e0 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f98c4001670 con 0x7f98cc0707e0 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 >> v1:172.21.15.40:6789/0 conn(0x7f98cc0707e0 legacy=0x7f98cc0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cb400640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b00039b0 0x7f98b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cb400640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b0004800 0x7f98b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cb400640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc09aca0 0x7f98b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0cdee0 con 0x7f98b0004800 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0cc110 con 0x7f98b00039b0 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b000b9c0 con 0x7f98cc09aca0 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cbe00640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b0004800 0x7f98b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98cbe00640 1 -- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc09aca0 msgr2=0x7f98b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98d1000640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b00039b0 0x7f98b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.156+0000 7f98d1a00640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc09aca0 0x7f98b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.157+0000 7f98cbe00640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc09aca0 0x7f98b00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.157+0000 7f98cbe00640 1 -- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b00039b0 msgr2=0x7f98b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.157+0000 7f98cbe00640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b00039b0 0x7f98b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.157+0000 7f98cbe00640 1 -- 172.21.15.40:0/3363456003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b0003520 con 0x7f98b0004800 2024-03-08T06:05:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.157+0000 7f98d1000640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b00039b0 0x7f98b0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:04.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.158+0000 7f98cbe00640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b0004800 0x7f98b0004c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009ac0 tx=0x7f98b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5eb0517d9b6908 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:04.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.158+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4039070 con 0x7f98b0004800 2024-03-08T06:05:04.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.158+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b402c6b0 con 0x7f98b0004800 2024-03-08T06:05:04.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.158+0000 7f98cb400640 1 -- 172.21.15.40:0/3363456003 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4009690 con 0x7f98b0004800 2024-03-08T06:05:04.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.158+0000 7f98d3a00640 1 -- 172.21.15.40:0/3363456003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b0004800 msgr2=0x7f98b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.158+0000 7f98d3a00640 1 --2- 172.21.15.40:0/3363456003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98b0004800 0x7f98b0004c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009ac0 tx=0x7f98b0005830 comp rx=0 tx=0).stop 2024-03-08T06:05:04.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.159+0000 7f98d1a00640 1 -- 172.21.15.40:0/3363456003 reap_dead start 2024-03-08T06:05:04.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.159+0000 7f98d3a00640 1 -- 172.21.15.40:0/3363456003 shutdown_connections 2024-03-08T06:05:04.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.159+0000 7f98d3a00640 1 -- 172.21.15.40:0/3363456003 >> 172.21.15.40:0/3363456003 conn(0x7f98cc0b3510 msgr2=0x7f98cc0b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:04.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.159+0000 7f98d3a00640 1 -- 172.21.15.40:0/3363456003 shutdown_connections 2024-03-08T06:05:04.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.159+0000 7f98d3a00640 1 -- 172.21.15.40:0/3363456003 wait complete. 2024-03-08T06:05:04.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.160+0000 7f98d3a00640 1 Processor -- start 2024-03-08T06:05:04.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.161+0000 7f98d3a00640 1 -- start start 2024-03-08T06:05:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d3a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0004800 0x7f98cc0b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d3a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc0caaf0 0x7f98cc0b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d3a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc0b73c0 0x7f98cc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d3a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc06ffe0 con 0x7f98cc0b73c0 2024-03-08T06:05:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d3a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0cc9b0 con 0x7f98b0004800 2024-03-08T06:05:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d3a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc06bd60 con 0x7f98cc0caaf0 2024-03-08T06:05:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d1a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc0b73c0 0x7f98cc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d1a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc0b73c0 0x7f98cc0b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53532/0 (socket says 172.21.15.40:53532) 2024-03-08T06:05:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d1a00640 1 -- 172.21.15.40:0/696693094 learned_addr learned my addr 172.21.15.40:0/696693094 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:04.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d1a00640 1 -- 172.21.15.40:0/696693094 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc0caaf0 msgr2=0x7f98cc0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d1a00640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc0caaf0 0x7f98cc0b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d1a00640 1 -- 172.21.15.40:0/696693094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0004800 msgr2=0x7f98cc0b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d1000640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0004800 0x7f98cc0b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d1a00640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0004800 0x7f98cc0b6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.162+0000 7f98d1a00640 1 -- 172.21.15.40:0/696693094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc160430 con 0x7f98cc0b73c0 2024-03-08T06:05:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.163+0000 7f98d1a00640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc0b73c0 0x7f98cc0b4db0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f98c4000c00 tx=0x7f98c400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.163+0000 7f98c8c00640 1 -- 172.21.15.40:0/696693094 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c40074e0 con 0x7f98cc0b73c0 2024-03-08T06:05:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.163+0000 7f98c8c00640 1 -- 172.21.15.40:0/696693094 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c40047e0 con 0x7f98cc0b73c0 2024-03-08T06:05:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.163+0000 7f98c8c00640 1 -- 172.21.15.40:0/696693094 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4004b90 con 0x7f98cc0b73c0 2024-03-08T06:05:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.163+0000 7f98d3a00640 1 -- 172.21.15.40:0/696693094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc0caf90 con 0x7f98cc0b73c0 2024-03-08T06:05:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.163+0000 7f98d3a00640 1 -- 172.21.15.40:0/696693094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc0cb550 con 0x7f98cc0b73c0 2024-03-08T06:05:04.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.164+0000 7f98d3a00640 1 -- 172.21.15.40:0/696693094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc067cb0 con 0x7f98cc0b73c0 2024-03-08T06:05:04.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.165+0000 7f98c8c00640 1 -- 172.21.15.40:0/696693094 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98c401b020 con 0x7f98cc0b73c0 2024-03-08T06:05:04.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.167+0000 7f98c8c00640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98a806b500 0x7f98a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.167+0000 7f98c8c00640 1 -- 172.21.15.40:0/696693094 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f98c40899f0 con 0x7f98cc0b73c0 2024-03-08T06:05:04.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.167+0000 7f98d1000640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98a806b500 0x7f98a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.168+0000 7f98d1000640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98a806b500 0x7f98a806d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f98bc007440 tx=0x7f98bc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.171+0000 7f98c8c00640 1 -- 172.21.15.40:0/696693094 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98c400f2f0 con 0x7f98cc0b73c0 2024-03-08T06:05:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.439+0000 7f98d3a00640 1 -- 172.21.15.40:0/696693094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f98cc0b78a0 con 0x7f98cc0b73c0 2024-03-08T06:05:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.440+0000 7f98c8c00640 1 -- 172.21.15.40:0/696693094 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f98c4058bb0 con 0x7f98cc0b73c0 2024-03-08T06:05:04.440 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:05:04.440 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:05:04.440 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-03-08T06:05:04.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.440 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-03-08T06:05:04.440 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAVqupl71wSLhAAS87lXSzQxwiWfqT/tiUZGw==", 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAWquplPVhRFhAACZ826lQeDP/KhF9bgKWZig==", 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.441 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAWqupllRL/AxAAeGht9vBbsgddcoXPRfRMVQ==", 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2024-03-08T06:05:04.442 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAWquplI7SRJxAAQ6ID823EOUIGkGMLNwUKpA==", 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAbquplz4pjIBAA3PPVSAKqCC9t+pxsnUE79Q==", 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAfquplnfWjDhAADHjcJSEAtOX7N6T7cIW3SA==", 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAiquplvTvCGBAA8t2oQZMt5Z+a5XE4qfS+Aw==", 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-03-08T06:05:04.444 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAlquplNvQCMhAArufflsQ9Qshoa5wxpB/xSg==", 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQArquplXMoPAhAApEEt2OjJynIXrPnGUY1+Pw==", 2024-03-08T06:05:04.445 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAuqupl5aafGxAAPRDg67jn/tZbqWzaYctCqQ==", 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAxqupltOB2JhAAFw4z1NKoHJWPC+uRJPLdmQ==", 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA0quply35uMRAAi3/Dr7awTdLwXmaRpdFHdg==", 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.447 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAWquplEsjANRAAl87CIlR0lmsGJba4MA4mdw==", 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAXquplxCwhEBAA2QmAZcA6M1tw4l8rvYrZpQ==", 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-03-08T06:05:04.448 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAXquplQzlPBRAAOkbXTAGJvuhC9xYAs+mrpQ==", 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.449 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAXquplB26ZGxAAPhP8P4DU1uK7K7HCaYfBRQ==", 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAUqupl3t8jHxAAT0ySW9i6aP0pfFYTLxvufQ==", 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-03-08T06:05:04.450 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mds", 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA+qupl7YuiEhAAMPidq5va8r+hJyob2dMjVQ==", 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mds" 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mgr", 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA+quplcqmiEhAAWRYurAYtj51tCzZYx5IacA==", 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.451 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-osd", 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA+quplhsKiEhAATyzeV7SxnseJzQs4jsP/3Q==", 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-osd" 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd", 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA+quplBNuiEhAAj0pe6aeaxpZ8FgT90wgEfw==", 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-08T06:05:04.452 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA+quplkPSiEhAAL9kt5XdZg+zuIitwm75Qyw==", 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rgw", 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA+qupllA6jEhAA0gEbF29Vq5ZAvhILBnE77w==", 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAVquplmyB7IRAA1QRz+cd43qLk8GikJZCAUg==", 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAVqupl2/Y1FBAACtDcILA/I2BmxOe83fcDXg==", 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-03-08T06:05:04.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-03-08T06:05:04.455 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.455 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:04.455 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-03-08T06:05:04.455 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 -- 172.21.15.40:0/696693094 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98a806b500 msgr2=0x7f98a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98a806b500 0x7f98a806d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f98bc007440 tx=0x7f98bc00b040 comp rx=0 tx=0).stop 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 -- 172.21.15.40:0/696693094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc0b73c0 msgr2=0x7f98cc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc0b73c0 0x7f98cc0b4db0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f98c4000c00 tx=0x7f98c400d9b0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 -- 172.21.15.40:0/696693094 shutdown_connections 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f98a806b500 0x7f98a806d9c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc0b73c0 0x7f98cc0b4db0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98cc0caaf0 0x7f98cc0b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 --2- 172.21.15.40:0/696693094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0004800 0x7f98cc0b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 -- 172.21.15.40:0/696693094 >> 172.21.15.40:0/696693094 conn(0x7f98cc0b3510 msgr2=0x7f98cc0ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 -- 172.21.15.40:0/696693094 shutdown_connections 2024-03-08T06:05:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.443+0000 7f98d3a00640 1 -- 172.21.15.40:0/696693094 wait complete. 2024-03-08T06:05:04.460 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-08T06:05:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.680+0000 7f324d800640 1 Processor -- start 2024-03-08T06:05:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.680+0000 7f324d800640 1 -- start start 2024-03-08T06:05:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.682+0000 7f324d800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f324806f050 0x7f3248071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.682+0000 7f324d800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3248071940 0x7f324815d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.682+0000 7f324d800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f324806d600 con 0x7f32481451b0 2024-03-08T06:05:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.682+0000 7f324d800640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3248071db0 con 0x7f3248071940 2024-03-08T06:05:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.682+0000 7f324d800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324806d8e0 con 0x7f324806f050 2024-03-08T06:05:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.683+0000 7f324ce00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3248071940 0x7f324815d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.683+0000 7f3247e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f32481451b0 0x7f324815ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48184/0 (socket says 172.21.15.40:48184) 2024-03-08T06:05:04.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.683+0000 7f3247e00640 1 -- 172.21.15.40:0/1120540472 learned_addr learned my addr 172.21.15.40:0/1120540472 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:04.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.683+0000 7f324ce00640 1 -- 172.21.15.40:0/1120540472 >> v2:172.21.15.53:3300/0 conn(0x7f324806f050 msgr2=0x7f3248071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:04.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.683+0000 7f323fe00640 1 --2- 172.21.15.40:0/1120540472 >> v2:172.21.15.53:3300/0 conn(0x7f324806f050 0x7f3248071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.683+0000 7f324ce00640 1 --2- 172.21.15.40:0/1120540472 >> v2:172.21.15.53:3300/0 conn(0x7f324806f050 0x7f3248071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.683+0000 7f324ce00640 1 -- 172.21.15.40:0/1120540472 >> v1:172.21.15.40:6789/0 conn(0x7f32481451b0 legacy=0x7f324815ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.683+0000 7f324ce00640 1 -- 172.21.15.40:0/1120540472 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324806dbc0 con 0x7f3248071940 2024-03-08T06:05:04.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.683+0000 7f3247400640 1 -- 172.21.15.40:0/1120540472 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992154094 0 0) 0x7f324806d600 con 0x7f32481451b0 2024-03-08T06:05:04.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.683+0000 7f323fe00640 1 --2- 172.21.15.40:0/1120540472 >> v2:172.21.15.53:3300/0 conn(0x7f324806f050 0x7f3248071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:04.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.684+0000 7f324ce00640 1 --2- 172.21.15.40:0/1120540472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3248071940 0x7f324815d1a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f32480a8660 tx=0x7f324002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b0372e7b308e1e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:04.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.685+0000 7f3247400640 1 -- 172.21.15.40:0/1120540472 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240039070 con 0x7f3248071940 2024-03-08T06:05:04.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.685+0000 7f3247400640 1 -- 172.21.15.40:0/1120540472 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324002cc70 con 0x7f3248071940 2024-03-08T06:05:04.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.685+0000 7f3247400640 1 -- 172.21.15.40:0/1120540472 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240034660 con 0x7f3248071940 2024-03-08T06:05:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.685+0000 7f324d800640 1 -- 172.21.15.40:0/1120540472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3248071940 msgr2=0x7f324815d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.685+0000 7f324d800640 1 --2- 172.21.15.40:0/1120540472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3248071940 0x7f324815d1a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f32480a8660 tx=0x7f324002bec0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.685+0000 7f324d800640 1 -- 172.21.15.40:0/1120540472 shutdown_connections 2024-03-08T06:05:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.685+0000 7f324d800640 1 --2- 172.21.15.40:0/1120540472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3248071940 0x7f324815d1a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.685+0000 7f324d800640 1 --2- 172.21.15.40:0/1120540472 >> v2:172.21.15.53:3300/0 conn(0x7f324806f050 0x7f3248071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.685+0000 7f324d800640 1 -- 172.21.15.40:0/1120540472 >> 172.21.15.40:0/1120540472 conn(0x7f32480b1590 msgr2=0x7f32480b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.685+0000 7f324d800640 1 -- 172.21.15.40:0/1120540472 shutdown_connections 2024-03-08T06:05:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.686+0000 7f324d800640 1 -- 172.21.15.40:0/1120540472 wait complete. 2024-03-08T06:05:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.687+0000 7f324d800640 1 Processor -- start 2024-03-08T06:05:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.687+0000 7f324d800640 1 -- start start 2024-03-08T06:05:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.688+0000 7f324d800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f324806f050 0x7f3248139c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.688+0000 7f324d800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3248071940 0x7f32480b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.688+0000 7f324d800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32481451b0 0x7f324815a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.688+0000 7f324d800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3248145710 con 0x7f324806f050 2024-03-08T06:05:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.688+0000 7f324d800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3248145590 con 0x7f32481451b0 2024-03-08T06:05:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.688+0000 7f324d800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324815d6e0 con 0x7f3248071940 2024-03-08T06:05:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.689+0000 7f3247e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f324806f050 0x7f3248139c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.689+0000 7f3247e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f324806f050 0x7f3248139c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53538/0 (socket says 172.21.15.40:53538) 2024-03-08T06:05:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.689+0000 7f3247e00640 1 -- 172.21.15.40:0/3961039339 learned_addr learned my addr 172.21.15.40:0/3961039339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.689+0000 7f323fe00640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3248071940 0x7f32480b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.689+0000 7f324ce00640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32481451b0 0x7f324815a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.689+0000 7f3247e00640 1 -- 172.21.15.40:0/3961039339 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3248071940 msgr2=0x7f32480b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.689+0000 7f3247e00640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3248071940 0x7f32480b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.689+0000 7f3247e00640 1 -- 172.21.15.40:0/3961039339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32481451b0 msgr2=0x7f324815a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.689+0000 7f3247e00640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32481451b0 0x7f324815a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.689+0000 7f3247e00640 1 -- 172.21.15.40:0/3961039339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324813c680 con 0x7f324806f050 2024-03-08T06:05:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.689+0000 7f3247e00640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f324806f050 0x7f3248139c40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f323800c460 tx=0x7f323800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.690+0000 7f3244c00640 1 -- 172.21.15.40:0/3961039339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238018070 con 0x7f324806f050 2024-03-08T06:05:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.690+0000 7f3244c00640 1 -- 172.21.15.40:0/3961039339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3238007500 con 0x7f324806f050 2024-03-08T06:05:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.690+0000 7f3244c00640 1 -- 172.21.15.40:0/3961039339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238004a20 con 0x7f324806f050 2024-03-08T06:05:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.690+0000 7f324d800640 1 -- 172.21.15.40:0/3961039339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324813c960 con 0x7f324806f050 2024-03-08T06:05:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.690+0000 7f324d800640 1 -- 172.21.15.40:0/3961039339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324813cf20 con 0x7f324806f050 2024-03-08T06:05:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.691+0000 7f324d800640 1 -- 172.21.15.40:0/3961039339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3248075090 con 0x7f324806f050 2024-03-08T06:05:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.692+0000 7f3244c00640 1 -- 172.21.15.40:0/3961039339 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3238004bc0 con 0x7f324806f050 2024-03-08T06:05:04.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.694+0000 7f3244c00640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f322406b4d0 0x7f322406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:04.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.694+0000 7f3244c00640 1 -- 172.21.15.40:0/3961039339 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3238089170 con 0x7f324806f050 2024-03-08T06:05:04.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.694+0000 7f323fe00640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f322406b4d0 0x7f322406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:04.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.695+0000 7f323fe00640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f322406b4d0 0x7f322406d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f32480cde90 tx=0x7f3230008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:04.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.698+0000 7f3244c00640 1 -- 172.21.15.40:0/3961039339 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3238058330 con 0x7f324806f050 2024-03-08T06:05:04.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.974+0000 7f324d800640 1 -- 172.21.15.40:0/3961039339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f32480cb870 con 0x7f324806f050 2024-03-08T06:05:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.978+0000 7f3244c00640 1 -- 172.21.15.40:0/3961039339 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3238057c30 con 0x7f324806f050 2024-03-08T06:05:04.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.981+0000 7f324d800640 1 -- 172.21.15.40:0/3961039339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f322406b4d0 msgr2=0x7f322406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.981+0000 7f324d800640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f322406b4d0 0x7f322406d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f32480cde90 tx=0x7f3230008040 comp rx=0 tx=0).stop 2024-03-08T06:05:04.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.981+0000 7f324d800640 1 -- 172.21.15.40:0/3961039339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f324806f050 msgr2=0x7f3248139c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:04.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.981+0000 7f324d800640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f324806f050 0x7f3248139c40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f323800c460 tx=0x7f323800c920 comp rx=0 tx=0).stop 2024-03-08T06:05:04.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.982+0000 7f324d800640 1 -- 172.21.15.40:0/3961039339 shutdown_connections 2024-03-08T06:05:04.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.982+0000 7f324d800640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f322406b4d0 0x7f322406d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.982+0000 7f324d800640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32481451b0 0x7f324815a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.982+0000 7f324d800640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3248071940 0x7f32480b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.982+0000 7f324d800640 1 --2- 172.21.15.40:0/3961039339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f324806f050 0x7f3248139c40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:04.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.982+0000 7f324d800640 1 -- 172.21.15.40:0/3961039339 >> 172.21.15.40:0/3961039339 conn(0x7f32480b1590 msgr2=0x7f324815c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:04.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.982+0000 7f324d800640 1 -- 172.21.15.40:0/3961039339 shutdown_connections 2024-03-08T06:05:04.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:04.982+0000 7f324d800640 1 -- 172.21.15.40:0/3961039339 wait complete. 2024-03-08T06:05:04.999 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-08T06:05:05.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.209+0000 7fc299a00640 1 Processor -- start 2024-03-08T06:05:05.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.209+0000 7fc299a00640 1 -- start start 2024-03-08T06:05:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.211+0000 7fc299a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2940cd6a0 0x7fc2940ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.211+0000 7fc299a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2940b3880 0x7fc2940cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.211+0000 7fc299a00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29415b290 con 0x7fc2940cd2c0 2024-03-08T06:05:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.211+0000 7fc299a00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2940cf7a0 con 0x7fc2940cd6a0 2024-03-08T06:05:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.211+0000 7fc299a00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29415b570 con 0x7fc2940b3880 2024-03-08T06:05:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.211+0000 7fc28be00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2940cd6a0 0x7fc2940ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.212+0000 7fc28be00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2940cd6a0 0x7fc2940ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53542/0 (socket says 172.21.15.40:53542) 2024-03-08T06:05:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.212+0000 7fc28be00640 1 -- 172.21.15.40:0/4077289067 learned_addr learned my addr 172.21.15.40:0/4077289067 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.212+0000 7fc293400640 1 --2- 172.21.15.40:0/4077289067 >> v2:172.21.15.53:3300/0 conn(0x7fc2940b3880 0x7fc2940cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.212+0000 7fc28be00640 1 -- 172.21.15.40:0/4077289067 >> v2:172.21.15.53:3300/0 conn(0x7fc2940b3880 msgr2=0x7fc2940cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:05.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.212+0000 7fc28be00640 1 --2- 172.21.15.40:0/4077289067 >> v2:172.21.15.53:3300/0 conn(0x7fc2940b3880 0x7fc2940cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.212+0000 7fc28be00640 1 -- 172.21.15.40:0/4077289067 >> v1:172.21.15.53:6789/0 conn(0x7fc2940cd2c0 legacy=0x7fc29406e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:05.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.212+0000 7fc28be00640 1 -- 172.21.15.40:0/4077289067 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29415b850 con 0x7fc2940cd6a0 2024-03-08T06:05:05.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.215+0000 7fc28be00640 1 --2- 172.21.15.40:0/4077289067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940cd6a0 0x7fc2940ced20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc2940b37a0 tx=0x7fc27c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e347a00d59ea4d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:05.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.215+0000 7fc292000640 1 -- 172.21.15.40:0/4077289067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c034070 con 0x7fc2940cd6a0 2024-03-08T06:05:05.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.215+0000 7fc292000640 1 -- 172.21.15.40:0/4077289067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27c030860 con 0x7fc2940cd6a0 2024-03-08T06:05:05.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.215+0000 7fc292000640 1 -- 172.21.15.40:0/4077289067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c030c30 con 0x7fc2940cd6a0 2024-03-08T06:05:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.216+0000 7fc299a00640 1 -- 172.21.15.40:0/4077289067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940cd6a0 msgr2=0x7fc2940ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.216+0000 7fc299a00640 1 --2- 172.21.15.40:0/4077289067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940cd6a0 0x7fc2940ced20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc2940b37a0 tx=0x7fc27c02b9a0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.216+0000 7fc299a00640 1 -- 172.21.15.40:0/4077289067 shutdown_connections 2024-03-08T06:05:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.216+0000 7fc299a00640 1 --2- 172.21.15.40:0/4077289067 >> v2:172.21.15.53:3300/0 conn(0x7fc2940b3880 0x7fc2940cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.216+0000 7fc299a00640 1 --2- 172.21.15.40:0/4077289067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940cd6a0 0x7fc2940ced20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.216+0000 7fc299a00640 1 -- 172.21.15.40:0/4077289067 >> 172.21.15.40:0/4077289067 conn(0x7fc29406ae30 msgr2=0x7fc29406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:05.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.216+0000 7fc299a00640 1 -- 172.21.15.40:0/4077289067 shutdown_connections 2024-03-08T06:05:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.216+0000 7fc299a00640 1 -- 172.21.15.40:0/4077289067 wait complete. 2024-03-08T06:05:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.218+0000 7fc299a00640 1 Processor -- start 2024-03-08T06:05:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.218+0000 7fc299a00640 1 -- start start 2024-03-08T06:05:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.219+0000 7fc299a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2940b3880 0x7fc294150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.219+0000 7fc299a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2940cd2c0 0x7fc294154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.219+0000 7fc299a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940cd6a0 0x7fc294155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.219+0000 7fc299a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29406eb60 con 0x7fc2940cd6a0 2024-03-08T06:05:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.219+0000 7fc299a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29415af40 con 0x7fc2940cd2c0 2024-03-08T06:05:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.219+0000 7fc299a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29406ece0 con 0x7fc2940b3880 2024-03-08T06:05:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.219+0000 7fc293400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940cd6a0 0x7fc294155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.219+0000 7fc293400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940cd6a0 0x7fc294155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53548/0 (socket says 172.21.15.40:53548) 2024-03-08T06:05:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.219+0000 7fc293400640 1 -- 172.21.15.40:0/1854532883 learned_addr learned my addr 172.21.15.40:0/1854532883 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.220+0000 7fc292a00640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2940b3880 0x7fc294150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:05.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.220+0000 7fc293400640 1 -- 172.21.15.40:0/1854532883 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2940b3880 msgr2=0x7fc294150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:05.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.220+0000 7fc293400640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2940b3880 0x7fc294150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.220+0000 7fc293400640 1 -- 172.21.15.40:0/1854532883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2940cd2c0 msgr2=0x7fc294154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:05.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.220+0000 7fc293400640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2940cd2c0 0x7fc294154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.220+0000 7fc293400640 1 -- 172.21.15.40:0/1854532883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc294153bc0 con 0x7fc2940cd6a0 2024-03-08T06:05:05.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.220+0000 7fc293400640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940cd6a0 0x7fc294155b10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00efc0 tx=0x7fc28c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:05.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.220+0000 7fc28aa00640 1 -- 172.21.15.40:0/1854532883 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c018020 con 0x7fc2940cd6a0 2024-03-08T06:05:05.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.220+0000 7fc28aa00640 1 -- 172.21.15.40:0/1854532883 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c014480 con 0x7fc2940cd6a0 2024-03-08T06:05:05.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.220+0000 7fc299a00640 1 -- 172.21.15.40:0/1854532883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc294153ea0 con 0x7fc2940cd6a0 2024-03-08T06:05:05.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.220+0000 7fc28aa00640 1 -- 172.21.15.40:0/1854532883 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c014870 con 0x7fc2940cd6a0 2024-03-08T06:05:05.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.221+0000 7fc299a00640 1 -- 172.21.15.40:0/1854532883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc294154460 con 0x7fc2940cd6a0 2024-03-08T06:05:05.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.222+0000 7fc299a00640 1 -- 172.21.15.40:0/1854532883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc294154950 con 0x7fc2940cd6a0 2024-03-08T06:05:05.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.223+0000 7fc28aa00640 1 -- 172.21.15.40:0/1854532883 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc28c005d10 con 0x7fc2940cd6a0 2024-03-08T06:05:05.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.224+0000 7fc28aa00640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc27006b4d0 0x7fc27006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.224+0000 7fc28aa00640 1 -- 172.21.15.40:0/1854532883 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc28c08a9a0 con 0x7fc2940cd6a0 2024-03-08T06:05:05.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.224+0000 7fc292a00640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc27006b4d0 0x7fc27006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:05.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.225+0000 7fc292a00640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc27006b4d0 0x7fc27006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc2840015a0 tx=0x7fc284009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:05.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.229+0000 7fc28aa00640 1 -- 172.21.15.40:0/1854532883 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc28c022070 con 0x7fc2940cd6a0 2024-03-08T06:05:05.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.497+0000 7fc299a00640 1 -- 172.21.15.40:0/1854532883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fc294067cb0 con 0x7fc2940cd6a0 2024-03-08T06:05:05.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.501+0000 7fc28aa00640 1 -- 172.21.15.40:0/1854532883 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc28c010040 con 0x7fc2940cd6a0 2024-03-08T06:05:05.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.504+0000 7fc299a00640 1 -- 172.21.15.40:0/1854532883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc27006b4d0 msgr2=0x7fc27006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:05.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.504+0000 7fc299a00640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc27006b4d0 0x7fc27006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc2840015a0 tx=0x7fc284009040 comp rx=0 tx=0).stop 2024-03-08T06:05:05.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.504+0000 7fc299a00640 1 -- 172.21.15.40:0/1854532883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940cd6a0 msgr2=0x7fc294155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:05.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.504+0000 7fc299a00640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940cd6a0 0x7fc294155b10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00efc0 tx=0x7fc28c00c490 comp rx=0 tx=0).stop 2024-03-08T06:05:05.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.504+0000 7fc299a00640 1 -- 172.21.15.40:0/1854532883 shutdown_connections 2024-03-08T06:05:05.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.505+0000 7fc299a00640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc27006b4d0 0x7fc27006d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.505+0000 7fc299a00640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940cd6a0 0x7fc294155b10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.505+0000 7fc299a00640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2940cd2c0 0x7fc294154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.505+0000 7fc299a00640 1 --2- 172.21.15.40:0/1854532883 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2940b3880 0x7fc294150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.505+0000 7fc299a00640 1 -- 172.21.15.40:0/1854532883 >> 172.21.15.40:0/1854532883 conn(0x7fc29406ae30 msgr2=0x7fc2940717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:05.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.505+0000 7fc299a00640 1 -- 172.21.15.40:0/1854532883 shutdown_connections 2024-03-08T06:05:05.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.505+0000 7fc299a00640 1 -- 172.21.15.40:0/1854532883 wait complete. 2024-03-08T06:05:05.524 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-08T06:05:05.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.750+0000 7f960d000640 1 Processor -- start 2024-03-08T06:05:05.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.750+0000 7f960d000640 1 -- start start 2024-03-08T06:05:05.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.752+0000 7f960d000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080b4420 0x7f96080b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.752+0000 7f960d000640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f96080ae1a0 0x7f96080ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.752+0000 7f960d000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960806ed10 con 0x7f96081442a0 2024-03-08T06:05:05.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.752+0000 7f960d000640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96080b4dc0 con 0x7f96080ae1a0 2024-03-08T06:05:05.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.752+0000 7f960d000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9608144680 con 0x7f96080b4420 2024-03-08T06:05:05.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.752+0000 7f95ffe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080b4420 0x7f96080b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:05.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.752+0000 7f9607e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f96080ae1a0 0x7f96080ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:05.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.753+0000 7f9607e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f96080ae1a0 0x7f96080ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:41948/0 (socket says 172.21.15.40:41948) 2024-03-08T06:05:05.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.753+0000 7f9607e00640 1 -- 172.21.15.40:0/1534952311 learned_addr learned my addr 172.21.15.40:0/1534952311 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:05.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.753+0000 7f95ffe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080b4420 0x7f96080b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:38126/0 (socket says 172.21.15.40:38126) 2024-03-08T06:05:05.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.753+0000 7f95ffe00640 1 -- 172.21.15.40:0/1534952311 >> v2:172.21.15.53:3300/0 conn(0x7f96080ae1a0 msgr2=0x7f96080ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.753+0000 7f95ffe00640 1 --2- 172.21.15.40:0/1534952311 >> v2:172.21.15.53:3300/0 conn(0x7f96080ae1a0 0x7f96080ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.753+0000 7f95ffe00640 1 -- 172.21.15.40:0/1534952311 >> v1:172.21.15.53:6789/0 conn(0x7f96081442a0 legacy=0x7f960806e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.753+0000 7f95ffe00640 1 -- 172.21.15.40:0/1534952311 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608161950 con 0x7f96080b4420 2024-03-08T06:05:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.753+0000 7f9607e00640 1 --2- 172.21.15.40:0/1534952311 >> v2:172.21.15.53:3300/0 conn(0x7f96080ae1a0 0x7f96080ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:05:05.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.755+0000 7f95ffe00640 1 --2- 172.21.15.40:0/1534952311 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080b4420 0x7f96080b4880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95f00097a0 tx=0x7f95f002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71cd4b84f2fea06c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:05.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.755+0000 7f9606a00640 1 -- 172.21.15.40:0/1534952311 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0035020 con 0x7f96080b4420 2024-03-08T06:05:05.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.756+0000 7f9606a00640 1 -- 172.21.15.40:0/1534952311 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f0031810 con 0x7f96080b4420 2024-03-08T06:05:05.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.756+0000 7f9606a00640 1 -- 172.21.15.40:0/1534952311 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0031be0 con 0x7f96080b4420 2024-03-08T06:05:05.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.756+0000 7f960d000640 1 -- 172.21.15.40:0/1534952311 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080b4420 msgr2=0x7f96080b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.756+0000 7f960d000640 1 --2- 172.21.15.40:0/1534952311 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080b4420 0x7f96080b4880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95f00097a0 tx=0x7f95f002b8b0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.756+0000 7f960d000640 1 -- 172.21.15.40:0/1534952311 shutdown_connections 2024-03-08T06:05:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.756+0000 7f960d000640 1 --2- 172.21.15.40:0/1534952311 >> v2:172.21.15.53:3300/0 conn(0x7f96080ae1a0 0x7f96080ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.756+0000 7f960d000640 1 --2- 172.21.15.40:0/1534952311 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080b4420 0x7f96080b4880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.756+0000 7f960d000640 1 -- 172.21.15.40:0/1534952311 >> 172.21.15.40:0/1534952311 conn(0x7f96080b1590 msgr2=0x7f96080b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.757+0000 7f960d000640 1 -- 172.21.15.40:0/1534952311 shutdown_connections 2024-03-08T06:05:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.757+0000 7f960d000640 1 -- 172.21.15.40:0/1534952311 wait complete. 2024-03-08T06:05:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.758+0000 7f960d000640 1 Processor -- start 2024-03-08T06:05:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.759+0000 7f960d000640 1 -- start start 2024-03-08T06:05:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.759+0000 7f960d000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080ae1a0 0x7f96081600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.759+0000 7f960d000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96080b4420 0x7f960806d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f960d000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96081442a0 0x7f960806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f960d000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96081621f0 con 0x7f96081442a0 2024-03-08T06:05:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f960d000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9608161f10 con 0x7f96080b4420 2024-03-08T06:05:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f960d000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96081624d0 con 0x7f96080ae1a0 2024-03-08T06:05:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f9607e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96081442a0 0x7f960806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f9607e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96081442a0 0x7f960806d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53556/0 (socket says 172.21.15.40:53556) 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f9607e00640 1 -- 172.21.15.40:0/1994349463 learned_addr learned my addr 172.21.15.40:0/1994349463 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f9607400640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080ae1a0 0x7f96081600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f9607e00640 1 -- 172.21.15.40:0/1994349463 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080ae1a0 msgr2=0x7f96081600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f95ffe00640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96080b4420 0x7f960806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f9607e00640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080ae1a0 0x7f96081600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f9607e00640 1 -- 172.21.15.40:0/1994349463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96080b4420 msgr2=0x7f960806d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f9607e00640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96080b4420 0x7f960806d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f9607e00640 1 -- 172.21.15.40:0/1994349463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96081f75e0 con 0x7f96081442a0 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.760+0000 7f9607400640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080ae1a0 0x7f96081600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.761+0000 7f9607e00640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96081442a0 0x7f960806d9a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f960000efc0 tx=0x7f960000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.761+0000 7f95ff400640 1 -- 172.21.15.40:0/1994349463 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960000cbe0 con 0x7f96081442a0 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.761+0000 7f95ff400640 1 -- 172.21.15.40:0/1994349463 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960000cd80 con 0x7f96081442a0 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.761+0000 7f960d000640 1 -- 172.21.15.40:0/1994349463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96080c4980 con 0x7f96081442a0 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.761+0000 7f95ff400640 1 -- 172.21.15.40:0/1994349463 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600003bb0 con 0x7f96081442a0 2024-03-08T06:05:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.761+0000 7f960d000640 1 -- 172.21.15.40:0/1994349463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96080c4f40 con 0x7f96081442a0 2024-03-08T06:05:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.762+0000 7f960d000640 1 -- 172.21.15.40:0/1994349463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9608075090 con 0x7f96081442a0 2024-03-08T06:05:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.763+0000 7f95ff400640 1 -- 172.21.15.40:0/1994349463 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f960001b030 con 0x7f96081442a0 2024-03-08T06:05:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.764+0000 7f95ff400640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f95e406b500 0x7f95e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.764+0000 7f95ff400640 1 -- 172.21.15.40:0/1994349463 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f960008a310 con 0x7f96081442a0 2024-03-08T06:05:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.765+0000 7f9607400640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f95e406b500 0x7f95e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.765+0000 7f9607400640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f95e406b500 0x7f95e406d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f95f80015e0 tx=0x7f95f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:05.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:05.769+0000 7f95ff400640 1 -- 172.21.15.40:0/1994349463 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9600022020 con 0x7f96081442a0 2024-03-08T06:05:06.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.044+0000 7f960d000640 1 -- 172.21.15.40:0/1994349463 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f960806c3e0 con 0x7f96081442a0 2024-03-08T06:05:06.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.049+0000 7f95ff400640 1 -- 172.21.15.40:0/1994349463 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9600010040 con 0x7f96081442a0 2024-03-08T06:05:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 -- 172.21.15.40:0/1994349463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f95e406b500 msgr2=0x7f95e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f95e406b500 0x7f95e406d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f95f80015e0 tx=0x7f95f8009040 comp rx=0 tx=0).stop 2024-03-08T06:05:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 -- 172.21.15.40:0/1994349463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96081442a0 msgr2=0x7f960806d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96081442a0 0x7f960806d9a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f960000efc0 tx=0x7f960000c490 comp rx=0 tx=0).stop 2024-03-08T06:05:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 -- 172.21.15.40:0/1994349463 shutdown_connections 2024-03-08T06:05:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f95e406b500 0x7f95e406d9c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96081442a0 0x7f960806d9a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96080b4420 0x7f960806d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 --2- 172.21.15.40:0/1994349463 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96080ae1a0 0x7f96081600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 -- 172.21.15.40:0/1994349463 >> 172.21.15.40:0/1994349463 conn(0x7f96080b1590 msgr2=0x7f96080b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 -- 172.21.15.40:0/1994349463 shutdown_connections 2024-03-08T06:05:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.052+0000 7f960d000640 1 -- 172.21.15.40:0/1994349463 wait complete. 2024-03-08T06:05:06.072 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-08T06:05:06.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.293+0000 7f7ddca00640 1 Processor -- start 2024-03-08T06:05:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.293+0000 7f7ddca00640 1 -- start start 2024-03-08T06:05:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.295+0000 7f7ddca00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7dd80aebe0 0x7f7dd8071c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.295+0000 7f7ddca00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7dd815e950 0x7f7dd80721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.295+0000 7f7ddca00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd8154bc0 con 0x7f7dd80b4380 2024-03-08T06:05:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.295+0000 7f7ddca00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd806ec90 con 0x7f7dd815e950 2024-03-08T06:05:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.295+0000 7f7ddca00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd80b4760 con 0x7f7dd80aebe0 2024-03-08T06:05:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.295+0000 7f7dd6800640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7dd80b4380 0x7f7dd815b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48198/0 (socket says 172.21.15.40:48198) 2024-03-08T06:05:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.296+0000 7f7dd6800640 1 -- 172.21.15.40:0/503042157 learned_addr learned my addr 172.21.15.40:0/503042157 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.296+0000 7f7dd7200640 1 --2- 172.21.15.40:0/503042157 >> v2:172.21.15.40:3300/0 conn(0x7f7dd815e950 0x7f7dd80721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.296+0000 7f7dcfe00640 1 --2- 172.21.15.40:0/503042157 >> v2:172.21.15.53:3300/0 conn(0x7f7dd80aebe0 0x7f7dd8071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.296+0000 7f7dd5e00640 1 -- 172.21.15.40:0/503042157 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661054654 0 0) 0x7f7dc8001110 con 0x7f7dd80b4380 2024-03-08T06:05:06.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.296+0000 7f7dd5e00640 1 -- 172.21.15.40:0/503042157 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dbc005730 con 0x7f7dd80b4380 2024-03-08T06:05:06.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.296+0000 7f7dd7200640 1 -- 172.21.15.40:0/503042157 >> v2:172.21.15.53:3300/0 conn(0x7f7dd80aebe0 msgr2=0x7f7dd8071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:06.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.296+0000 7f7dd7200640 1 --2- 172.21.15.40:0/503042157 >> v2:172.21.15.53:3300/0 conn(0x7f7dd80aebe0 0x7f7dd8071c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.296+0000 7f7dd7200640 1 -- 172.21.15.40:0/503042157 >> v1:172.21.15.40:6789/0 conn(0x7f7dd80b4380 legacy=0x7f7dd815b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.296+0000 7f7dd7200640 1 -- 172.21.15.40:0/503042157 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd80af080 con 0x7f7dd815e950 2024-03-08T06:05:06.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.297+0000 7f7dd7200640 1 --2- 172.21.15.40:0/503042157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dd815e950 0x7f7dd80721d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80b2120 tx=0x7f7dd002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452264ff95a13dde server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:06.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.298+0000 7f7dd5e00640 1 -- 172.21.15.40:0/503042157 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd002ca70 con 0x7f7dd815e950 2024-03-08T06:05:06.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.298+0000 7f7dd5e00640 1 -- 172.21.15.40:0/503042157 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd002cc10 con 0x7f7dd815e950 2024-03-08T06:05:06.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.298+0000 7f7dd5e00640 1 -- 172.21.15.40:0/503042157 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0008440 con 0x7f7dd815e950 2024-03-08T06:05:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.298+0000 7f7ddca00640 1 -- 172.21.15.40:0/503042157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dd815e950 msgr2=0x7f7dd80721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.298+0000 7f7ddca00640 1 --2- 172.21.15.40:0/503042157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dd815e950 0x7f7dd80721d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80b2120 tx=0x7f7dd002bed0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.298+0000 7f7ddca00640 1 -- 172.21.15.40:0/503042157 shutdown_connections 2024-03-08T06:05:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.298+0000 7f7ddca00640 1 --2- 172.21.15.40:0/503042157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dd815e950 0x7f7dd80721d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.298+0000 7f7ddca00640 1 --2- 172.21.15.40:0/503042157 >> v2:172.21.15.53:3300/0 conn(0x7f7dd80aebe0 0x7f7dd8071c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.298+0000 7f7ddca00640 1 -- 172.21.15.40:0/503042157 >> 172.21.15.40:0/503042157 conn(0x7f7dd80b1590 msgr2=0x7f7dd80b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.298+0000 7f7ddca00640 1 -- 172.21.15.40:0/503042157 shutdown_connections 2024-03-08T06:05:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.298+0000 7f7ddca00640 1 -- 172.21.15.40:0/503042157 wait complete. 2024-03-08T06:05:06.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.300+0000 7f7ddca00640 1 Processor -- start 2024-03-08T06:05:06.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.300+0000 7f7ddca00640 1 -- start start 2024-03-08T06:05:06.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.301+0000 7f7ddca00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dd80aebe0 0x7f7dd8152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:06.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.301+0000 7f7ddca00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7dd80b4380 0x7f7dd81531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:06.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.301+0000 7f7ddca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dd815e950 0x7f7dd81f0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:06.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.301+0000 7f7ddca00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd815e150 con 0x7f7dd80aebe0 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.301+0000 7f7ddca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd80af360 con 0x7f7dd815e950 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.301+0000 7f7ddca00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd815e430 con 0x7f7dd80b4380 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.301+0000 7f7dd6800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dd80aebe0 0x7f7dd8152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.301+0000 7f7dd6800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dd80aebe0 0x7f7dd8152c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53572/0 (socket says 172.21.15.40:53572) 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.301+0000 7f7dd6800640 1 -- 172.21.15.40:0/2622537598 learned_addr learned my addr 172.21.15.40:0/2622537598 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.302+0000 7f7dcfe00640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7dd80b4380 0x7f7dd81531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.302+0000 7f7dd7200640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dd815e950 0x7f7dd81f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.302+0000 7f7dd6800640 1 -- 172.21.15.40:0/2622537598 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7dd80b4380 msgr2=0x7f7dd81531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.302+0000 7f7dd6800640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7dd80b4380 0x7f7dd81531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.302+0000 7f7dd6800640 1 -- 172.21.15.40:0/2622537598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dd815e950 msgr2=0x7f7dd81f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.302+0000 7f7dd6800640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dd815e950 0x7f7dd81f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.302+0000 7f7dd6800640 1 -- 172.21.15.40:0/2622537598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8159310 con 0x7f7dd80aebe0 2024-03-08T06:05:06.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.302+0000 7f7dd7200640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dd815e950 0x7f7dd81f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:06.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.302+0000 7f7dd6800640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dd80aebe0 0x7f7dd8152c70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c730 tx=0x7f7dc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:06.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.303+0000 7f7dcea00640 1 -- 172.21.15.40:0/2622537598 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8017070 con 0x7f7dd80aebe0 2024-03-08T06:05:06.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.303+0000 7f7dcea00640 1 -- 172.21.15.40:0/2622537598 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc8005ad0 con 0x7f7dd80aebe0 2024-03-08T06:05:06.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.303+0000 7f7dcea00640 1 -- 172.21.15.40:0/2622537598 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8007e20 con 0x7f7dd80aebe0 2024-03-08T06:05:06.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.303+0000 7f7ddca00640 1 -- 172.21.15.40:0/2622537598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd81552f0 con 0x7f7dd80aebe0 2024-03-08T06:05:06.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.303+0000 7f7ddca00640 1 -- 172.21.15.40:0/2622537598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd81558b0 con 0x7f7dd80aebe0 2024-03-08T06:05:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.304+0000 7f7ddca00640 1 -- 172.21.15.40:0/2622537598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd8075090 con 0x7f7dd80aebe0 2024-03-08T06:05:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.305+0000 7f7dcea00640 1 -- 172.21.15.40:0/2622537598 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7dc801a070 con 0x7f7dd80aebe0 2024-03-08T06:05:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.306+0000 7f7dcea00640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7db406b500 0x7f7db406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.306+0000 7f7dcea00640 1 -- 172.21.15.40:0/2622537598 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7dc8089320 con 0x7f7dd80aebe0 2024-03-08T06:05:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.306+0000 7f7dcfe00640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7db406b500 0x7f7db406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:06.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.307+0000 7f7dcfe00640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7db406b500 0x7f7db406d9c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80afc10 tx=0x7f7dc0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:06.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.310+0000 7f7dcea00640 1 -- 172.21.15.40:0/2622537598 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7dd8154bc0 con 0x7f7dd80aebe0 2024-03-08T06:05:06.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.579+0000 7f7ddca00640 1 -- 172.21.15.40:0/2622537598 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f7dd8158dd0 con 0x7f7dd80aebe0 2024-03-08T06:05:06.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.585+0000 7f7dcea00640 1 -- 172.21.15.40:0/2622537598 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7dc8058450 con 0x7f7dd80aebe0 2024-03-08T06:05:06.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.587+0000 7f7ddca00640 1 -- 172.21.15.40:0/2622537598 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7db406b500 msgr2=0x7f7db406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.587+0000 7f7ddca00640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7db406b500 0x7f7db406d9c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80afc10 tx=0x7f7dc0008040 comp rx=0 tx=0).stop 2024-03-08T06:05:06.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.587+0000 7f7ddca00640 1 -- 172.21.15.40:0/2622537598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dd80aebe0 msgr2=0x7f7dd8152c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.588+0000 7f7ddca00640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dd80aebe0 0x7f7dd8152c70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c730 tx=0x7f7dc800cbf0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.588+0000 7f7ddca00640 1 -- 172.21.15.40:0/2622537598 shutdown_connections 2024-03-08T06:05:06.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.588+0000 7f7ddca00640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7db406b500 0x7f7db406d9c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.588+0000 7f7ddca00640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dd815e950 0x7f7dd81f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.588+0000 7f7ddca00640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7dd80b4380 0x7f7dd81531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.588+0000 7f7ddca00640 1 --2- 172.21.15.40:0/2622537598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dd80aebe0 0x7f7dd8152c70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.588+0000 7f7ddca00640 1 -- 172.21.15.40:0/2622537598 >> 172.21.15.40:0/2622537598 conn(0x7f7dd80b1590 msgr2=0x7f7dd8071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:06.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.588+0000 7f7ddca00640 1 -- 172.21.15.40:0/2622537598 shutdown_connections 2024-03-08T06:05:06.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.588+0000 7f7ddca00640 1 -- 172.21.15.40:0/2622537598 wait complete. 2024-03-08T06:05:06.606 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-08T06:05:06.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.823+0000 7f057c400640 1 Processor -- start 2024-03-08T06:05:06.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.824+0000 7f057c400640 1 -- start start 2024-03-08T06:05:06.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.825+0000 7f057c400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f057414bbe0 0x7f05740cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.825+0000 7f057c400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0574166530 con 0x7f05740ce150 2024-03-08T06:05:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.825+0000 7f057c400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0574166810 con 0x7f057414b560 2024-03-08T06:05:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.825+0000 7f057c400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05740b2dc0 con 0x7f057414bbe0 2024-03-08T06:05:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.826+0000 7f0579000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f057414bbe0 0x7f05740cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.826+0000 7f0579000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f057414bbe0 0x7f05740cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53578/0 (socket says 172.21.15.40:53578) 2024-03-08T06:05:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.826+0000 7f0579000640 1 -- 172.21.15.40:0/3785831803 learned_addr learned my addr 172.21.15.40:0/3785831803 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.826+0000 7f0579000640 1 -- 172.21.15.40:0/3785831803 >> v1:172.21.15.53:6789/0 conn(0x7f057414b560 legacy=0x7f0574165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.826+0000 7f0579000640 1 -- 172.21.15.40:0/3785831803 >> v1:172.21.15.40:6789/0 conn(0x7f05740ce150 legacy=0x7f05740717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.826+0000 7f0579000640 1 -- 172.21.15.40:0/3785831803 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0574166af0 con 0x7f057414bbe0 2024-03-08T06:05:06.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.827+0000 7f0579000640 1 --2- 172.21.15.40:0/3785831803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f057414bbe0 0x7f05740cdc10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f05740b2f40 tx=0x7f056002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1cbe6901c2980a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:06.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.828+0000 7f0573e00640 1 -- 172.21.15.40:0/3785831803 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560034070 con 0x7f057414bbe0 2024-03-08T06:05:06.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.828+0000 7f0573e00640 1 -- 172.21.15.40:0/3785831803 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0560030840 con 0x7f057414bbe0 2024-03-08T06:05:06.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.828+0000 7f0573e00640 1 -- 172.21.15.40:0/3785831803 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560030c10 con 0x7f057414bbe0 2024-03-08T06:05:06.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.828+0000 7f057c400640 1 -- 172.21.15.40:0/3785831803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f057414bbe0 msgr2=0x7f05740cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.828+0000 7f057c400640 1 --2- 172.21.15.40:0/3785831803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f057414bbe0 0x7f05740cdc10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f05740b2f40 tx=0x7f056002b980 comp rx=0 tx=0).stop 2024-03-08T06:05:06.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.828+0000 7f057c400640 1 -- 172.21.15.40:0/3785831803 shutdown_connections 2024-03-08T06:05:06.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.828+0000 7f057c400640 1 --2- 172.21.15.40:0/3785831803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f057414bbe0 0x7f05740cdc10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.828+0000 7f057c400640 1 -- 172.21.15.40:0/3785831803 >> 172.21.15.40:0/3785831803 conn(0x7f057406ae30 msgr2=0x7f057406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:06.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.828+0000 7f057c400640 1 -- 172.21.15.40:0/3785831803 shutdown_connections 2024-03-08T06:05:06.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.828+0000 7f057c400640 1 -- 172.21.15.40:0/3785831803 wait complete. 2024-03-08T06:05:06.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.830+0000 7f057c400640 1 Processor -- start 2024-03-08T06:05:06.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.830+0000 7f057c400640 1 -- start start 2024-03-08T06:05:06.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.831+0000 7f057c400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f05740ce150 0x7f057415f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:06.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.831+0000 7f057c400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f057414b560 0x7f05741ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:06.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.831+0000 7f057c400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f057414bbe0 0x7f05741651e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:06.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.831+0000 7f057c400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0574071ee0 con 0x7f057414bbe0 2024-03-08T06:05:06.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.831+0000 7f057c400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057414b940 con 0x7f057414b560 2024-03-08T06:05:06.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.831+0000 7f057c400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0574072060 con 0x7f05740ce150 2024-03-08T06:05:06.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.831+0000 7f057a400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f057414bbe0 0x7f05741651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:06.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.831+0000 7f057a400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f057414bbe0 0x7f05741651e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53592/0 (socket says 172.21.15.40:53592) 2024-03-08T06:05:06.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.831+0000 7f057a400640 1 -- 172.21.15.40:0/2180130975 learned_addr learned my addr 172.21.15.40:0/2180130975 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:06.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f0579000640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f057414b560 0x7f05741ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:06.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f0579a00640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f05740ce150 0x7f057415f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:06.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f057a400640 1 -- 172.21.15.40:0/2180130975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f05740ce150 msgr2=0x7f057415f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f057a400640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f05740ce150 0x7f057415f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f057a400640 1 -- 172.21.15.40:0/2180130975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f057414b560 msgr2=0x7f05741ebb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f057a400640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f057414b560 0x7f05741ebb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f057a400640 1 -- 172.21.15.40:0/2180130975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0574147bd0 con 0x7f057414bbe0 2024-03-08T06:05:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f0579a00640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f05740ce150 0x7f057415f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f0579000640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f057414b560 0x7f05741ebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f057a400640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f057414bbe0 0x7f05741651e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f056c00c760 tx=0x7f056c00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f0571600640 1 -- 172.21.15.40:0/2180130975 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c016020 con 0x7f057414bbe0 2024-03-08T06:05:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f0571600640 1 -- 172.21.15.40:0/2180130975 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c006d50 con 0x7f057414bbe0 2024-03-08T06:05:06.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f0571600640 1 -- 172.21.15.40:0/2180130975 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c003bb0 con 0x7f057414bbe0 2024-03-08T06:05:06.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.832+0000 7f057c400640 1 -- 172.21.15.40:0/2180130975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0574147eb0 con 0x7f057414bbe0 2024-03-08T06:05:06.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.833+0000 7f057c400640 1 -- 172.21.15.40:0/2180130975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0574148470 con 0x7f057414bbe0 2024-03-08T06:05:06.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.834+0000 7f057c400640 1 -- 172.21.15.40:0/2180130975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0574067cb0 con 0x7f057414bbe0 2024-03-08T06:05:06.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.835+0000 7f0571600640 1 -- 172.21.15.40:0/2180130975 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f056c0050d0 con 0x7f057414bbe0 2024-03-08T06:05:06.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.835+0000 7f0571600640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f055406b4d0 0x7f055406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:06.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.836+0000 7f0571600640 1 -- 172.21.15.40:0/2180130975 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f056c089ab0 con 0x7f057414bbe0 2024-03-08T06:05:06.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.836+0000 7f0579a00640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f055406b4d0 0x7f055406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:06.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.836+0000 7f0579a00640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f055406b4d0 0x7f055406d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f056400a830 tx=0x7f0564008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:06.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:06.840+0000 7f0571600640 1 -- 172.21.15.40:0/2180130975 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f056c020020 con 0x7f057414bbe0 2024-03-08T06:05:07.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.114+0000 7f057c400640 1 -- 172.21.15.40:0/2180130975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f057406eb10 con 0x7f057414bbe0 2024-03-08T06:05:07.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.119+0000 7f0571600640 1 -- 172.21.15.40:0/2180130975 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f056c058c70 con 0x7f057414bbe0 2024-03-08T06:05:07.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.121+0000 7f057c400640 1 -- 172.21.15.40:0/2180130975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f055406b4d0 msgr2=0x7f055406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.121+0000 7f057c400640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f055406b4d0 0x7f055406d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f056400a830 tx=0x7f0564008040 comp rx=0 tx=0).stop 2024-03-08T06:05:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.121+0000 7f057c400640 1 -- 172.21.15.40:0/2180130975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f057414bbe0 msgr2=0x7f05741651e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.121+0000 7f057c400640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f057414bbe0 0x7f05741651e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f056c00c760 tx=0x7f056c00cb60 comp rx=0 tx=0).stop 2024-03-08T06:05:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.122+0000 7f057c400640 1 -- 172.21.15.40:0/2180130975 shutdown_connections 2024-03-08T06:05:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.122+0000 7f057c400640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f055406b4d0 0x7f055406d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.122+0000 7f057c400640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f057414bbe0 0x7f05741651e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.122+0000 7f057c400640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f057414b560 0x7f05741ebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.122+0000 7f057c400640 1 --2- 172.21.15.40:0/2180130975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f05740ce150 0x7f057415f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.122+0000 7f057c400640 1 -- 172.21.15.40:0/2180130975 >> 172.21.15.40:0/2180130975 conn(0x7f057406ae30 msgr2=0x7f05740ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:07.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.122+0000 7f057c400640 1 -- 172.21.15.40:0/2180130975 shutdown_connections 2024-03-08T06:05:07.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.122+0000 7f057c400640 1 -- 172.21.15.40:0/2180130975 wait complete. 2024-03-08T06:05:07.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-08T06:05:07.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.358+0000 7fa9ea200640 1 Processor -- start 2024-03-08T06:05:07.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.358+0000 7fa9ea200640 1 -- start start 2024-03-08T06:05:07.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.359+0000 7fa9ea200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9e41451b0 0x7fa9e415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.359+0000 7fa9ea200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e415cf70 con 0x7fa9e4071b10 2024-03-08T06:05:07.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.359+0000 7fa9ea200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e406d600 con 0x7fa9e406f050 2024-03-08T06:05:07.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.359+0000 7fa9ea200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e406d8e0 con 0x7fa9e41451b0 2024-03-08T06:05:07.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.360+0000 7fa9e9800640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa9e4071b10 0x7fa9e415d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43364/0 (socket says 172.21.15.40:43364) 2024-03-08T06:05:07.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.360+0000 7fa9e9800640 1 -- 172.21.15.40:0/3178558808 learned_addr learned my addr 172.21.15.40:0/3178558808 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:07.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.360+0000 7fa9e8e00640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9e41451b0 0x7fa9e415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.360+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892372632 0 0) 0x7fa9e415cf70 con 0x7fa9e4071b10 2024-03-08T06:05:07.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.360+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c8003520 con 0x7fa9e4071b10 2024-03-08T06:05:07.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 747220629 0 0) 0x7fa9c8003520 con 0x7fa9e4071b10 2024-03-08T06:05:07.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9e41451b0 msgr2=0x7fa9e415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:07.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9e41451b0 0x7fa9e415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 >> v1:172.21.15.53:6789/0 conn(0x7fa9e406f050 legacy=0x7fa9e4071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e406dbc0 con 0x7fa9e4071b10 2024-03-08T06:05:07.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fa9dc001670 con 0x7fa9e4071b10 2024-03-08T06:05:07.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 >> v1:172.21.15.40:6789/0 conn(0x7fa9e4071b10 legacy=0x7fa9e415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9e41451b0 0x7fa9c8003ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9c8004090 0x7fa9c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c8004d90 0x7fa9c8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e406d8e0 con 0x7fa9c8004090 2024-03-08T06:05:07.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c800bc00 con 0x7fa9c8004d90 2024-03-08T06:05:07.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c800be00 con 0x7fa9e41451b0 2024-03-08T06:05:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e3e00640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9c8004090 0x7fa9c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.361+0000 7fa9e8e00640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9e41451b0 0x7fa9c8003ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.362+0000 7fa9e9800640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c8004d90 0x7fa9c8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.362+0000 7fa9e3e00640 1 -- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9e41451b0 msgr2=0x7fa9c8003ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.362+0000 7fa9e3e00640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9e41451b0 0x7fa9c8003ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.362+0000 7fa9e3e00640 1 -- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c8004d90 msgr2=0x7fa9c8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.362+0000 7fa9e3e00640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c8004d90 0x7fa9c8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.362+0000 7fa9e3e00640 1 -- 172.21.15.40:0/3178558808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c8003520 con 0x7fa9c8004090 2024-03-08T06:05:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.362+0000 7fa9e9800640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c8004d90 0x7fa9c8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:07.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.363+0000 7fa9e3e00640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9c8004090 0x7fa9c8004800 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0001e20 tx=0x7fa9c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f48bb8e8301288d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.363+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0001a70 con 0x7fa9c8004090 2024-03-08T06:05:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.363+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0003b70 con 0x7fa9c8004090 2024-03-08T06:05:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.363+0000 7fa9e3400640 1 -- 172.21.15.40:0/3178558808 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00050d0 con 0x7fa9c8004090 2024-03-08T06:05:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.364+0000 7fa9ea200640 1 -- 172.21.15.40:0/3178558808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9c8004090 msgr2=0x7fa9c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.364+0000 7fa9ea200640 1 --2- 172.21.15.40:0/3178558808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9c8004090 0x7fa9c8004800 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0001e20 tx=0x7fa9c8005830 comp rx=0 tx=0).stop 2024-03-08T06:05:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.364+0000 7fa9e9800640 1 -- 172.21.15.40:0/3178558808 reap_dead start 2024-03-08T06:05:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.364+0000 7fa9ea200640 1 -- 172.21.15.40:0/3178558808 shutdown_connections 2024-03-08T06:05:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.364+0000 7fa9ea200640 1 -- 172.21.15.40:0/3178558808 >> 172.21.15.40:0/3178558808 conn(0x7fa9e40b1590 msgr2=0x7fa9e40b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.364+0000 7fa9ea200640 1 -- 172.21.15.40:0/3178558808 shutdown_connections 2024-03-08T06:05:07.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.364+0000 7fa9ea200640 1 -- 172.21.15.40:0/3178558808 wait complete. 2024-03-08T06:05:07.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.366+0000 7fa9ea200640 1 Processor -- start 2024-03-08T06:05:07.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.366+0000 7fa9ea200640 1 -- start start 2024-03-08T06:05:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.367+0000 7fa9ea200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9c8004090 0x7fa9e40adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.367+0000 7fa9ea200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e415b2f0 0x7fa9e4141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.367+0000 7fa9ea200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e415a730 0x7fa9e415abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.367+0000 7fa9ea200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e4145710 con 0x7fa9e415a730 2024-03-08T06:05:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.367+0000 7fa9ea200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e4145590 con 0x7fa9e415b2f0 2024-03-08T06:05:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.367+0000 7fa9ea200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e415ca30 con 0x7fa9c8004090 2024-03-08T06:05:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.367+0000 7fa9e9800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e415a730 0x7fa9e415abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.367+0000 7fa9e9800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e415a730 0x7fa9e415abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33418/0 (socket says 172.21.15.40:33418) 2024-03-08T06:05:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.367+0000 7fa9e9800640 1 -- 172.21.15.40:0/2078516768 learned_addr learned my addr 172.21.15.40:0/2078516768 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.368+0000 7fa9e8e00640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9c8004090 0x7fa9e40adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.368+0000 7fa9e3e00640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e415b2f0 0x7fa9e4141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.368+0000 7fa9e9800640 1 -- 172.21.15.40:0/2078516768 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9c8004090 msgr2=0x7fa9e40adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.368+0000 7fa9e9800640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9c8004090 0x7fa9e40adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.368+0000 7fa9e9800640 1 -- 172.21.15.40:0/2078516768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e415b2f0 msgr2=0x7fa9e4141fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.368+0000 7fa9e9800640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e415b2f0 0x7fa9e4141fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.368+0000 7fa9e9800640 1 -- 172.21.15.40:0/2078516768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e415b0f0 con 0x7fa9e415a730 2024-03-08T06:05:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.368+0000 7fa9e9800640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e415a730 0x7fa9e415abb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc000fb0 tx=0x7fa9dc00d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.368+0000 7fa9e0c00640 1 -- 172.21.15.40:0/2078516768 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0074e0 con 0x7fa9e415a730 2024-03-08T06:05:07.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.369+0000 7fa9e0c00640 1 -- 172.21.15.40:0/2078516768 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc0047e0 con 0x7fa9e415a730 2024-03-08T06:05:07.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.369+0000 7fa9e0c00640 1 -- 172.21.15.40:0/2078516768 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc004bd0 con 0x7fa9e415a730 2024-03-08T06:05:07.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.369+0000 7fa9ea200640 1 -- 172.21.15.40:0/2078516768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e40ae820 con 0x7fa9e415a730 2024-03-08T06:05:07.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.369+0000 7fa9ea200640 1 -- 172.21.15.40:0/2078516768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e413bd40 con 0x7fa9e415a730 2024-03-08T06:05:07.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.370+0000 7fa9ea200640 1 -- 172.21.15.40:0/2078516768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e4075090 con 0x7fa9e415a730 2024-03-08T06:05:07.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.371+0000 7fa9e0c00640 1 -- 172.21.15.40:0/2078516768 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9dc014430 con 0x7fa9e415a730 2024-03-08T06:05:07.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.372+0000 7fa9e0c00640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9c406b500 0x7fa9c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.372+0000 7fa9e0c00640 1 -- 172.21.15.40:0/2078516768 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa9dc089660 con 0x7fa9e415a730 2024-03-08T06:05:07.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.372+0000 7fa9e8e00640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9c406b500 0x7fa9c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.373+0000 7fa9e8e00640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9c406b500 0x7fa9c406d9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009a10 tx=0x7fa9d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:07.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.377+0000 7fa9e0c00640 1 -- 172.21.15.40:0/2078516768 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9e415b0f0 con 0x7fa9e415a730 2024-03-08T06:05:07.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.651+0000 7fa9ea200640 1 -- 172.21.15.40:0/2078516768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa9e40cf140 con 0x7fa9e415a730 2024-03-08T06:05:07.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.656+0000 7fa9e0c00640 1 -- 172.21.15.40:0/2078516768 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa9dc0588c0 con 0x7fa9e415a730 2024-03-08T06:05:07.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.659+0000 7fa9ea200640 1 -- 172.21.15.40:0/2078516768 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9c406b500 msgr2=0x7fa9c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.659+0000 7fa9ea200640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9c406b500 0x7fa9c406d9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009a10 tx=0x7fa9d4008040 comp rx=0 tx=0).stop 2024-03-08T06:05:07.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.659+0000 7fa9ea200640 1 -- 172.21.15.40:0/2078516768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e415a730 msgr2=0x7fa9e415abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.659+0000 7fa9ea200640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e415a730 0x7fa9e415abb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc000fb0 tx=0x7fa9dc00d950 comp rx=0 tx=0).stop 2024-03-08T06:05:07.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.659+0000 7fa9ea200640 1 -- 172.21.15.40:0/2078516768 shutdown_connections 2024-03-08T06:05:07.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.659+0000 7fa9ea200640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa9c406b500 0x7fa9c406d9c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.659+0000 7fa9ea200640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e415a730 0x7fa9e415abb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.659+0000 7fa9ea200640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9e415b2f0 0x7fa9e4141fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.659+0000 7fa9ea200640 1 --2- 172.21.15.40:0/2078516768 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa9c8004090 0x7fa9e40adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.659+0000 7fa9ea200640 1 -- 172.21.15.40:0/2078516768 >> 172.21.15.40:0/2078516768 conn(0x7fa9e40b1590 msgr2=0x7fa9e415ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:07.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.659+0000 7fa9ea200640 1 -- 172.21.15.40:0/2078516768 shutdown_connections 2024-03-08T06:05:07.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.660+0000 7fa9ea200640 1 -- 172.21.15.40:0/2078516768 wait complete. 2024-03-08T06:05:07.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:05:07.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.894+0000 7f3804600640 1 Processor -- start 2024-03-08T06:05:07.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.895+0000 7f3804600640 1 -- start start 2024-03-08T06:05:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.896+0000 7f3804600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37fc15c7d0 0x7f37fc163160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.896+0000 7f3804600640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37fc06e280 0x7f37fc071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.896+0000 7f3804600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc1636a0 con 0x7f37fc0b46a0 2024-03-08T06:05:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.896+0000 7f3804600640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc06ddf0 con 0x7f37fc06e280 2024-03-08T06:05:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.896+0000 7f3804600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc15cbb0 con 0x7f37fc15c7d0 2024-03-08T06:05:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.897+0000 7f3801200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37fc06e280 0x7f37fc071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.897+0000 7f3801200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37fc06e280 0x7f37fc071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33424/0 (socket says 172.21.15.40:33424) 2024-03-08T06:05:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.897+0000 7f3801200640 1 -- 172.21.15.40:0/2233993448 learned_addr learned my addr 172.21.15.40:0/2233993448 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.897+0000 7f3802600640 1 --1- 172.21.15.40:0/2233993448 >> v1:172.21.15.40:6789/0 conn(0x7f37fc0b46a0 0x7f37fc165cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43372/0 (socket says 172.21.15.40:43372) 2024-03-08T06:05:07.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.897+0000 7f3801c00640 1 --2- 172.21.15.40:0/2233993448 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37fc15c7d0 0x7f37fc163160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.897+0000 7f3801200640 1 -- 172.21.15.40:0/2233993448 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37fc15c7d0 msgr2=0x7f37fc163160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:07.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.897+0000 7f3801200640 1 --2- 172.21.15.40:0/2233993448 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37fc15c7d0 0x7f37fc163160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.897+0000 7f3801200640 1 -- 172.21.15.40:0/2233993448 >> v1:172.21.15.40:6789/0 conn(0x7f37fc0b46a0 legacy=0x7f37fc165cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.897+0000 7f3801200640 1 -- 172.21.15.40:0/2233993448 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc0b4c40 con 0x7f37fc06e280 2024-03-08T06:05:07.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.897+0000 7f37fbe00640 1 -- 172.21.15.40:0/2233993448 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310118893 0 0) 0x7f37f4001110 con 0x7f37fc0b46a0 2024-03-08T06:05:07.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.899+0000 7f3801200640 1 --2- 172.21.15.40:0/2233993448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37fc06e280 0x7f37fc071840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009870 tx=0x7f37e802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36053a574a5e5e81 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:07.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.899+0000 7f37fbe00640 1 -- 172.21.15.40:0/2233993448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8034070 con 0x7f37fc06e280 2024-03-08T06:05:07.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.899+0000 7f37fbe00640 1 -- 172.21.15.40:0/2233993448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e8030840 con 0x7f37fc06e280 2024-03-08T06:05:07.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.899+0000 7f37fbe00640 1 -- 172.21.15.40:0/2233993448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8030c10 con 0x7f37fc06e280 2024-03-08T06:05:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.899+0000 7f3804600640 1 -- 172.21.15.40:0/2233993448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37fc06e280 msgr2=0x7f37fc071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.899+0000 7f3804600640 1 --2- 172.21.15.40:0/2233993448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37fc06e280 0x7f37fc071840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009870 tx=0x7f37e802b980 comp rx=0 tx=0).stop 2024-03-08T06:05:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.900+0000 7f3804600640 1 -- 172.21.15.40:0/2233993448 shutdown_connections 2024-03-08T06:05:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.900+0000 7f3804600640 1 --2- 172.21.15.40:0/2233993448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37fc06e280 0x7f37fc071840 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.900+0000 7f3804600640 1 --2- 172.21.15.40:0/2233993448 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37fc15c7d0 0x7f37fc163160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.900+0000 7f3804600640 1 -- 172.21.15.40:0/2233993448 >> 172.21.15.40:0/2233993448 conn(0x7f37fc06ae30 msgr2=0x7f37fc06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.900+0000 7f3804600640 1 -- 172.21.15.40:0/2233993448 shutdown_connections 2024-03-08T06:05:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.900+0000 7f3804600640 1 -- 172.21.15.40:0/2233993448 wait complete. 2024-03-08T06:05:07.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.901+0000 7f3804600640 1 Processor -- start 2024-03-08T06:05:07.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.902+0000 7f3804600640 1 -- start start 2024-03-08T06:05:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.902+0000 7f3804600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37fc06e280 0x7f37fc160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.902+0000 7f3804600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37fc0b46a0 0x7f37fc1613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3804600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37fc15c7d0 0x7f37fc15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3804600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc071d80 con 0x7f37fc06e280 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3804600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc06da80 con 0x7f37fc0b46a0 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3804600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc071f00 con 0x7f37fc15c7d0 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3801c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37fc06e280 0x7f37fc160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3801c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37fc06e280 0x7f37fc160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33430/0 (socket says 172.21.15.40:33430) 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3801c00640 1 -- 172.21.15.40:0/3176469120 learned_addr learned my addr 172.21.15.40:0/3176469120 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3801c00640 1 -- 172.21.15.40:0/3176469120 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37fc15c7d0 msgr2=0x7f37fc15ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3801c00640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37fc15c7d0 0x7f37fc15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3801200640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37fc0b46a0 0x7f37fc1613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3801c00640 1 -- 172.21.15.40:0/3176469120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37fc0b46a0 msgr2=0x7f37fc1613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3801c00640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37fc0b46a0 0x7f37fc1613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3801c00640 1 -- 172.21.15.40:0/3176469120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc15f210 con 0x7f37fc06e280 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.903+0000 7f3801200640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37fc0b46a0 0x7f37fc1613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.904+0000 7f3801c00640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37fc06e280 0x7f37fc160530 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f37ec007e70 tx=0x7f37ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.904+0000 7f37f9600640 1 -- 172.21.15.40:0/3176469120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec014620 con 0x7f37fc06e280 2024-03-08T06:05:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.904+0000 7f37f9600640 1 -- 172.21.15.40:0/3176469120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec019070 con 0x7f37fc06e280 2024-03-08T06:05:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.904+0000 7f3804600640 1 -- 172.21.15.40:0/3176469120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc15f490 con 0x7f37fc06e280 2024-03-08T06:05:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.904+0000 7f37f9600640 1 -- 172.21.15.40:0/3176469120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec014a00 con 0x7f37fc06e280 2024-03-08T06:05:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.904+0000 7f3804600640 1 -- 172.21.15.40:0/3176469120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc15fa50 con 0x7f37fc06e280 2024-03-08T06:05:07.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.905+0000 7f3804600640 1 -- 172.21.15.40:0/3176469120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37fc067cb0 con 0x7f37fc06e280 2024-03-08T06:05:07.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.906+0000 7f37f9600640 1 -- 172.21.15.40:0/3176469120 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37ec014ba0 con 0x7f37fc06e280 2024-03-08T06:05:07.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.907+0000 7f37f9600640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f37dc06b4d0 0x7f37dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:07.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.907+0000 7f37f9600640 1 -- 172.21.15.40:0/3176469120 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f37ec08a850 con 0x7f37fc06e280 2024-03-08T06:05:07.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.907+0000 7f3801200640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f37dc06b4d0 0x7f37dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:07.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.908+0000 7f3801200640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f37dc06b4d0 0x7f37dc06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f37e8036000 tx=0x7f37e803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:07.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:07.911+0000 7f37f9600640 1 -- 172.21.15.40:0/3176469120 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37fc15f210 con 0x7f37fc06e280 2024-03-08T06:05:08.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.102+0000 7f3804600640 1 -- 172.21.15.40:0/3176469120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37fc0b3680 con 0x7f37fc06e280 2024-03-08T06:05:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.105+0000 7f37f9600640 1 -- 172.21.15.40:0/3176469120 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12807 (secure 0 0 0) 0x7f37ec0598f0 con 0x7f37fc06e280 2024-03-08T06:05:08.105 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:05:08.105 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:05:03.926656+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:05:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 -- 172.21.15.40:0/3176469120 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f37dc06b4d0 msgr2=0x7f37dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f37dc06b4d0 0x7f37dc06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f37e8036000 tx=0x7f37e803b040 comp rx=0 tx=0).stop 2024-03-08T06:05:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 -- 172.21.15.40:0/3176469120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37fc06e280 msgr2=0x7f37fc160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37fc06e280 0x7f37fc160530 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f37ec007e70 tx=0x7f37ec00e4b0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 -- 172.21.15.40:0/3176469120 shutdown_connections 2024-03-08T06:05:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f37dc06b4d0 0x7f37dc06d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f37fc15c7d0 0x7f37fc15ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37fc0b46a0 0x7f37fc1613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 --2- 172.21.15.40:0/3176469120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37fc06e280 0x7f37fc160530 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 -- 172.21.15.40:0/3176469120 >> 172.21.15.40:0/3176469120 conn(0x7f37fc06ae30 msgr2=0x7f37fc0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 -- 172.21.15.40:0/3176469120 shutdown_connections 2024-03-08T06:05:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.107+0000 7f3804600640 1 -- 172.21.15.40:0/3176469120 wait complete. 2024-03-08T06:05:08.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:05:08.125 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:05:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.339+0000 7f7bfe800640 1 Processor -- start 2024-03-08T06:05:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.340+0000 7f7bfe800640 1 -- start start 2024-03-08T06:05:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.341+0000 7f7bfe800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7bf8072760 0x7f7bf806d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.341+0000 7f7bfe800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf80ae650 con 0x7f7bf8071a60 2024-03-08T06:05:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.341+0000 7f7bfe800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf80aee60 con 0x7f7bf8071e40 2024-03-08T06:05:08.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.341+0000 7f7bfe800640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf80ad460 con 0x7f7bf8072760 2024-03-08T06:05:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.341+0000 7f7bf6a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7bf8072760 0x7f7bf806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.342+0000 7f7bf7400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7bf8071a60 0x7f7bf80adf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43388/0 (socket says 172.21.15.40:43388) 2024-03-08T06:05:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.342+0000 7f7bf7400640 1 -- 172.21.15.40:0/822108579 learned_addr learned my addr 172.21.15.40:0/822108579 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.342+0000 7f7bf6a00640 1 -- 172.21.15.40:0/822108579 >> v1:172.21.15.53:6789/0 conn(0x7f7bf8071e40 legacy=0x7f7bf80b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.342+0000 7f7bf6a00640 1 -- 172.21.15.40:0/822108579 >> v1:172.21.15.40:6789/0 conn(0x7f7bf8071a60 legacy=0x7f7bf80adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.342+0000 7f7bf6000640 1 -- 172.21.15.40:0/822108579 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197619282 0 0) 0x7f7bf80ae650 con 0x7f7bf8071a60 2024-03-08T06:05:08.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.342+0000 7f7bf6a00640 1 -- 172.21.15.40:0/822108579 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf80ad740 con 0x7f7bf8072760 2024-03-08T06:05:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.344+0000 7f7bf6a00640 1 --2- 172.21.15.40:0/822108579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bf8072760 0x7f7bf806d850 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7be00097a0 tx=0x7f7be002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bfa288ad93e046 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.344+0000 7f7bf6000640 1 -- 172.21.15.40:0/822108579 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be00315a0 con 0x7f7bf8072760 2024-03-08T06:05:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.344+0000 7f7bf6000640 1 -- 172.21.15.40:0/822108579 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be0037070 con 0x7f7bf8072760 2024-03-08T06:05:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.344+0000 7f7bf6000640 1 -- 172.21.15.40:0/822108579 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0031960 con 0x7f7bf8072760 2024-03-08T06:05:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.344+0000 7f7bfe800640 1 -- 172.21.15.40:0/822108579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bf8072760 msgr2=0x7f7bf806d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.344+0000 7f7bfe800640 1 --2- 172.21.15.40:0/822108579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bf8072760 0x7f7bf806d850 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7be00097a0 tx=0x7f7be002b840 comp rx=0 tx=0).stop 2024-03-08T06:05:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.345+0000 7f7bfe800640 1 -- 172.21.15.40:0/822108579 shutdown_connections 2024-03-08T06:05:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.345+0000 7f7bfe800640 1 --2- 172.21.15.40:0/822108579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bf8072760 0x7f7bf806d850 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.345+0000 7f7bfe800640 1 -- 172.21.15.40:0/822108579 >> 172.21.15.40:0/822108579 conn(0x7f7bf80cfaa0 msgr2=0x7f7bf80cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.345+0000 7f7bfe800640 1 -- 172.21.15.40:0/822108579 shutdown_connections 2024-03-08T06:05:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.345+0000 7f7bfe800640 1 -- 172.21.15.40:0/822108579 wait complete. 2024-03-08T06:05:08.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.347+0000 7f7bfe800640 1 Processor -- start 2024-03-08T06:05:08.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.347+0000 7f7bfe800640 1 -- start start 2024-03-08T06:05:08.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.348+0000 7f7bfe800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7bf8071a60 0x7f7bf815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.348+0000 7f7bfe800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bf8071e40 0x7f7bf81f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.348+0000 7f7bfe800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf8072760 0x7f7bf81f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.348+0000 7f7bfe800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf80ad110 con 0x7f7bf8071e40 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.348+0000 7f7bfe800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf80aeb10 con 0x7f7bf8072760 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.348+0000 7f7bfe800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf80b3260 con 0x7f7bf8071a60 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.349+0000 7f7bf6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bf8071e40 0x7f7bf81f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.349+0000 7f7bf6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bf8071e40 0x7f7bf81f6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33440/0 (socket says 172.21.15.40:33440) 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.349+0000 7f7bf6a00640 1 -- 172.21.15.40:0/388617220 learned_addr learned my addr 172.21.15.40:0/388617220 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.349+0000 7f7bf7400640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7bf8071a60 0x7f7bf815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.349+0000 7f7bf7e00640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf8072760 0x7f7bf81f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.349+0000 7f7bf6a00640 1 -- 172.21.15.40:0/388617220 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7bf8071a60 msgr2=0x7f7bf815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.349+0000 7f7bf6a00640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7bf8071a60 0x7f7bf815ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.349+0000 7f7bf6a00640 1 -- 172.21.15.40:0/388617220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf8072760 msgr2=0x7f7bf81f67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.349+0000 7f7bf6a00640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf8072760 0x7f7bf81f67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.349+0000 7f7bf6a00640 1 -- 172.21.15.40:0/388617220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf80c4da0 con 0x7f7bf8071e40 2024-03-08T06:05:08.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.349+0000 7f7bf7e00640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf8072760 0x7f7bf81f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:08.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.350+0000 7f7bf6a00640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bf8071e40 0x7f7bf81f6280 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7be00097a0 tx=0x7f7be00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:08.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.350+0000 7f7bef400640 1 -- 172.21.15.40:0/388617220 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0004030 con 0x7f7bf8071e40 2024-03-08T06:05:08.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.350+0000 7f7bef400640 1 -- 172.21.15.40:0/388617220 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be00041f0 con 0x7f7bf8071e40 2024-03-08T06:05:08.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.350+0000 7f7bef400640 1 -- 172.21.15.40:0/388617220 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0032640 con 0x7f7bf8071e40 2024-03-08T06:05:08.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.350+0000 7f7bfe800640 1 -- 172.21.15.40:0/388617220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf81f4f40 con 0x7f7bf8071e40 2024-03-08T06:05:08.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.350+0000 7f7bfe800640 1 -- 172.21.15.40:0/388617220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf81f57a0 con 0x7f7bf8071e40 2024-03-08T06:05:08.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.351+0000 7f7bfe800640 1 -- 172.21.15.40:0/388617220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf8067cb0 con 0x7f7bf8071e40 2024-03-08T06:05:08.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.352+0000 7f7bef400640 1 -- 172.21.15.40:0/388617220 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7be00327e0 con 0x7f7bf8071e40 2024-03-08T06:05:08.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.353+0000 7f7bef400640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7bd406b4d0 0x7f7bd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.353+0000 7f7bef400640 1 -- 172.21.15.40:0/388617220 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7be002f030 con 0x7f7bf8071e40 2024-03-08T06:05:08.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.353+0000 7f7bf7400640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7bd406b4d0 0x7f7bd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.354+0000 7f7bf7400640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7bd406b4d0 0x7f7bd406d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7be8000f60 tx=0x7f7be8007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:08.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.357+0000 7f7bef400640 1 -- 172.21.15.40:0/388617220 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7be0041070 con 0x7f7bf8071e40 2024-03-08T06:05:08.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.557+0000 7f7bfe800640 1 -- 172.21.15.40:0/388617220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bf80cfe80 con 0x7f7bf8071e40 2024-03-08T06:05:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.558+0000 7f7bef400640 1 -- 172.21.15.40:0/388617220 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7be007b3e0 con 0x7f7bf8071e40 2024-03-08T06:05:08.559 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:05:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.561+0000 7f7bfe800640 1 -- 172.21.15.40:0/388617220 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7bd406b4d0 msgr2=0x7f7bd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.561+0000 7f7bfe800640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7bd406b4d0 0x7f7bd406d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7be8000f60 tx=0x7f7be8007b50 comp rx=0 tx=0).stop 2024-03-08T06:05:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.561+0000 7f7bfe800640 1 -- 172.21.15.40:0/388617220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bf8071e40 msgr2=0x7f7bf81f6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.561+0000 7f7bfe800640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bf8071e40 0x7f7bf81f6280 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7be00097a0 tx=0x7f7be00046c0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.561+0000 7f7bfe800640 1 -- 172.21.15.40:0/388617220 shutdown_connections 2024-03-08T06:05:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.561+0000 7f7bfe800640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7bd406b4d0 0x7f7bd406d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.561+0000 7f7bfe800640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bf8072760 0x7f7bf81f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.561+0000 7f7bfe800640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bf8071e40 0x7f7bf81f6280 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.561+0000 7f7bfe800640 1 --2- 172.21.15.40:0/388617220 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7bf8071a60 0x7f7bf815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.561+0000 7f7bfe800640 1 -- 172.21.15.40:0/388617220 >> 172.21.15.40:0/388617220 conn(0x7f7bf80cfaa0 msgr2=0x7f7bf80af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.562+0000 7f7bfe800640 1 -- 172.21.15.40:0/388617220 shutdown_connections 2024-03-08T06:05:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.562+0000 7f7bfe800640 1 -- 172.21.15.40:0/388617220 wait complete. 2024-03-08T06:05:08.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-08T06:05:08.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.801+0000 7fe1e0200640 1 Processor -- start 2024-03-08T06:05:08.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.801+0000 7fe1e0200640 1 -- start start 2024-03-08T06:05:08.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.802+0000 7fe1e0200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1d815f3a0 0x7fe1d809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.802+0000 7fe1e0200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d815f780 con 0x7fe1d80d05e0 2024-03-08T06:05:08.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.802+0000 7fe1e0200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d80cc600 con 0x7fe1d80ce120 2024-03-08T06:05:08.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.802+0000 7fe1e0200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d80cc8e0 con 0x7fe1d815f3a0 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.803+0000 7fe1de200640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe1d80d05e0 0x7fe1d80c55d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43394/0 (socket says 172.21.15.40:43394) 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.803+0000 7fe1de200640 1 -- 172.21.15.40:0/1011110401 learned_addr learned my addr 172.21.15.40:0/1011110401 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.803+0000 7fe1dd800640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1d815f3a0 0x7fe1d809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.803+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880247403 0 0) 0x7fe1d815f780 con 0x7fe1d80d05e0 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.803+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1c0003520 con 0x7fe1d80d05e0 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2242936446 0 0) 0x7fe1c0003520 con 0x7fe1d80d05e0 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1d815f3a0 msgr2=0x7fe1d809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1d815f3a0 0x7fe1d809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 >> v1:172.21.15.53:6789/0 conn(0x7fe1d80ce120 legacy=0x7fe1d80cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d80ccbc0 con 0x7fe1d80d05e0 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fe1d00019f0 con 0x7fe1d80d05e0 2024-03-08T06:05:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 >> v1:172.21.15.40:6789/0 conn(0x7fe1d80d05e0 legacy=0x7fe1d80c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1d815f3a0 0x7fe1c00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1c0003fb0 0x7fe1c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1c0004d40 0x7fe1c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d80cc8e0 con 0x7fe1c0004d40 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c000bb00 con 0x7fe1c0003fb0 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c000bd00 con 0x7fe1d815f3a0 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.804+0000 7fe1de200640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1c0004d40 0x7fe1c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.805+0000 7fe1dd800640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1d815f3a0 0x7fe1c00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.805+0000 7fe1dce00640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1c0003fb0 0x7fe1c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.805+0000 7fe1de200640 1 -- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1d815f3a0 msgr2=0x7fe1c00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.805+0000 7fe1de200640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1d815f3a0 0x7fe1c00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.805+0000 7fe1de200640 1 -- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1c0003fb0 msgr2=0x7fe1c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.805+0000 7fe1de200640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1c0003fb0 0x7fe1c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.805+0000 7fe1de200640 1 -- 172.21.15.40:0/1011110401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c0003520 con 0x7fe1c0004d40 2024-03-08T06:05:08.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.805+0000 7fe1dce00640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1c0003fb0 0x7fe1c0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:08.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.806+0000 7fe1de200640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1c0004d40 0x7fe1c00051c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe1d809b5e0 tx=0x7fe1c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d337789470e570a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.806+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d000a3d0 con 0x7fe1c0004d40 2024-03-08T06:05:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.806+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d0034630 con 0x7fe1c0004d40 2024-03-08T06:05:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.806+0000 7fe1d7e00640 1 -- 172.21.15.40:0/1011110401 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0034aa0 con 0x7fe1c0004d40 2024-03-08T06:05:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.807+0000 7fe1e0200640 1 -- 172.21.15.40:0/1011110401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1c0004d40 msgr2=0x7fe1c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.807+0000 7fe1e0200640 1 --2- 172.21.15.40:0/1011110401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1c0004d40 0x7fe1c00051c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe1d809b5e0 tx=0x7fe1c0005830 comp rx=0 tx=0).stop 2024-03-08T06:05:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.807+0000 7fe1de200640 1 -- 172.21.15.40:0/1011110401 reap_dead start 2024-03-08T06:05:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.807+0000 7fe1e0200640 1 -- 172.21.15.40:0/1011110401 shutdown_connections 2024-03-08T06:05:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.807+0000 7fe1e0200640 1 -- 172.21.15.40:0/1011110401 >> 172.21.15.40:0/1011110401 conn(0x7fe1d80d1a80 msgr2=0x7fe1d80d1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.807+0000 7fe1e0200640 1 -- 172.21.15.40:0/1011110401 shutdown_connections 2024-03-08T06:05:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.807+0000 7fe1e0200640 1 -- 172.21.15.40:0/1011110401 wait complete. 2024-03-08T06:05:08.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.809+0000 7fe1e0200640 1 Processor -- start 2024-03-08T06:05:08.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.809+0000 7fe1e0200640 1 -- start start 2024-03-08T06:05:08.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.810+0000 7fe1e0200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1c0004d40 0x7fe1d815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.810+0000 7fe1e0200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1d80b6510 0x7fe1d80b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.810+0000 7fe1e0200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d8151cf0 0x7fe1d80cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.810+0000 7fe1e0200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d80cf650 con 0x7fe1d80b6510 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.810+0000 7fe1e0200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d80cfac0 con 0x7fe1d8151cf0 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.810+0000 7fe1e0200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d80cf7d0 con 0x7fe1c0004d40 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.810+0000 7fe1dce00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1d80b6510 0x7fe1d80b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.810+0000 7fe1dce00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1d80b6510 0x7fe1d80b7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33454/0 (socket says 172.21.15.40:33454) 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.810+0000 7fe1dce00640 1 -- 172.21.15.40:0/4281966754 learned_addr learned my addr 172.21.15.40:0/4281966754 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.810+0000 7fe1dd800640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1c0004d40 0x7fe1d815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.810+0000 7fe1de200640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d8151cf0 0x7fe1d80cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1dce00640 1 -- 172.21.15.40:0/4281966754 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1c0004d40 msgr2=0x7fe1d815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1dce00640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1c0004d40 0x7fe1d815b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1dce00640 1 -- 172.21.15.40:0/4281966754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d8151cf0 msgr2=0x7fe1d80cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1dce00640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d8151cf0 0x7fe1d80cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1dce00640 1 -- 172.21.15.40:0/4281966754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d80b5470 con 0x7fe1d80b6510 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1de200640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d8151cf0 0x7fe1d80cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1dce00640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1d80b6510 0x7fe1d80b7380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00d7d0 tx=0x7fe1cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1d5600640 1 -- 172.21.15.40:0/4281966754 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc005070 con 0x7fe1d80b6510 2024-03-08T06:05:08.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1d5600640 1 -- 172.21.15.40:0/4281966754 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc005230 con 0x7fe1d80b6510 2024-03-08T06:05:08.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1d5600640 1 -- 172.21.15.40:0/4281966754 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc007030 con 0x7fe1d80b6510 2024-03-08T06:05:08.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1e0200640 1 -- 172.21.15.40:0/4281966754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d80b5750 con 0x7fe1d80b6510 2024-03-08T06:05:08.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.811+0000 7fe1e0200640 1 -- 172.21.15.40:0/4281966754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d80b5d10 con 0x7fe1d80b6510 2024-03-08T06:05:08.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.812+0000 7fe1e0200640 1 -- 172.21.15.40:0/4281966754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d80683c0 con 0x7fe1d80b6510 2024-03-08T06:05:08.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.813+0000 7fe1d5600640 1 -- 172.21.15.40:0/4281966754 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1cc01a020 con 0x7fe1d80b6510 2024-03-08T06:05:08.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.815+0000 7fe1d5600640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe1bc06b500 0x7fe1bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:08.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.815+0000 7fe1d5600640 1 -- 172.21.15.40:0/4281966754 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe1cc089800 con 0x7fe1d80b6510 2024-03-08T06:05:08.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.815+0000 7fe1dd800640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe1bc06b500 0x7fe1bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:08.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.816+0000 7fe1dd800640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe1bc06b500 0x7fe1bc06d9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009a10 tx=0x7fe1c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:08.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:08.819+0000 7fe1d5600640 1 -- 172.21.15.40:0/4281966754 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1cc058960 con 0x7fe1d80b6510 2024-03-08T06:05:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.019+0000 7fe1e0200640 1 -- 172.21.15.40:0/4281966754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe1d80d27e0 con 0x7fe1d80b6510 2024-03-08T06:05:09.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.019+0000 7fe1d5600640 1 -- 172.21.15.40:0/4281966754 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe1cc058260 con 0x7fe1d80b6510 2024-03-08T06:05:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:05:09.020 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-03-08T06:05:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-03-08T06:05:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-03-08T06:05:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:09.020 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-03-08T06:05:09.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 -- 172.21.15.40:0/4281966754 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe1bc06b500 msgr2=0x7fe1bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:09.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe1bc06b500 0x7fe1bc06d9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009a10 tx=0x7fe1c8008040 comp rx=0 tx=0).stop 2024-03-08T06:05:09.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 -- 172.21.15.40:0/4281966754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1d80b6510 msgr2=0x7fe1d80b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:09.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1d80b6510 0x7fe1d80b7380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00d7d0 tx=0x7fe1cc00dc90 comp rx=0 tx=0).stop 2024-03-08T06:05:09.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 -- 172.21.15.40:0/4281966754 shutdown_connections 2024-03-08T06:05:09.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe1bc06b500 0x7fe1bc06d9c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d8151cf0 0x7fe1d80cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1d80b6510 0x7fe1d80b7380 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 --2- 172.21.15.40:0/4281966754 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe1c0004d40 0x7fe1d815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 -- 172.21.15.40:0/4281966754 >> 172.21.15.40:0/4281966754 conn(0x7fe1d80d1a80 msgr2=0x7fe1d806fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 -- 172.21.15.40:0/4281966754 shutdown_connections 2024-03-08T06:05:09.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.022+0000 7fe1e0200640 1 -- 172.21.15.40:0/4281966754 wait complete. 2024-03-08T06:05:09.041 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-08T06:05:09.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-08T06:05:09.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.268+0000 7f0757000640 1 Processor -- start 2024-03-08T06:05:09.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.269+0000 7f0757000640 1 -- start start 2024-03-08T06:05:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.270+0000 7f0757000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f075006d520 0x7f0750071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.270+0000 7f0757000640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0750071a60 0x7f07500cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:09.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.270+0000 7f0757000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f075006dac0 con 0x7f0750144770 2024-03-08T06:05:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.270+0000 7f0757000640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f075006dda0 con 0x7f0750071a60 2024-03-08T06:05:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.270+0000 7f0757000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075006e080 con 0x7f075006d520 2024-03-08T06:05:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.271+0000 7f0756600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0750071a60 0x7f07500cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.271+0000 7f0756600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0750071a60 0x7f07500cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:35978/0 (socket says 172.21.15.40:35978) 2024-03-08T06:05:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.271+0000 7f0756600640 1 -- 172.21.15.40:0/2860866454 learned_addr learned my addr 172.21.15.40:0/2860866454 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:09.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.271+0000 7f0755200640 1 --2- 172.21.15.40:0/2860866454 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f075006d520 0x7f0750071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.272+0000 7f0756600640 1 -- 172.21.15.40:0/2860866454 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f075006d520 msgr2=0x7f0750071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.272+0000 7f0756600640 1 --2- 172.21.15.40:0/2860866454 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f075006d520 0x7f0750071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.272+0000 7f0756600640 1 -- 172.21.15.40:0/2860866454 >> v1:172.21.15.53:6789/0 conn(0x7f0750144770 legacy=0x7f07501451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:09.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.272+0000 7f0756600640 1 -- 172.21.15.40:0/2860866454 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075006e360 con 0x7f0750071a60 2024-03-08T06:05:09.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.273+0000 7f0756600640 1 --2- 172.21.15.40:0/2860866454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0750071a60 0x7f07500cc7f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0748009e50 tx=0x7f074802bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11ca8db8cb12f661 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:09.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.274+0000 7f074fe00640 1 -- 172.21.15.40:0/2860866454 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074802ca80 con 0x7f0750071a60 2024-03-08T06:05:09.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.274+0000 7f074fe00640 1 -- 172.21.15.40:0/2860866454 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074802cc20 con 0x7f0750071a60 2024-03-08T06:05:09.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.274+0000 7f074fe00640 1 -- 172.21.15.40:0/2860866454 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07480343f0 con 0x7f0750071a60 2024-03-08T06:05:09.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.275+0000 7f0757000640 1 -- 172.21.15.40:0/2860866454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0750071a60 msgr2=0x7f07500cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:09.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.275+0000 7f0757000640 1 --2- 172.21.15.40:0/2860866454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0750071a60 0x7f07500cc7f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0748009e50 tx=0x7f074802bee0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.275+0000 7f0757000640 1 -- 172.21.15.40:0/2860866454 shutdown_connections 2024-03-08T06:05:09.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.275+0000 7f0757000640 1 --2- 172.21.15.40:0/2860866454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0750071a60 0x7f07500cc7f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.275+0000 7f0757000640 1 --2- 172.21.15.40:0/2860866454 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f075006d520 0x7f0750071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.275+0000 7f0757000640 1 -- 172.21.15.40:0/2860866454 >> 172.21.15.40:0/2860866454 conn(0x7f075006ac50 msgr2=0x7f075006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:09.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.275+0000 7f0757000640 1 -- 172.21.15.40:0/2860866454 shutdown_connections 2024-03-08T06:05:09.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.275+0000 7f0757000640 1 -- 172.21.15.40:0/2860866454 wait complete. 2024-03-08T06:05:09.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.277+0000 7f0757000640 1 Processor -- start 2024-03-08T06:05:09.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.278+0000 7f0757000640 1 -- start start 2024-03-08T06:05:09.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.279+0000 7f0757000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f075006d520 0x7f075015b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:09.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.279+0000 7f0757000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0750071a60 0x7f07501e9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:09.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.279+0000 7f0757000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0750144770 0x7f07501e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:09.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.279+0000 7f0757000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07500cceb0 con 0x7f075006d520 2024-03-08T06:05:09.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.279+0000 7f0757000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07500ccd30 con 0x7f0750071a60 2024-03-08T06:05:09.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.279+0000 7f0757000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07500cd030 con 0x7f0750144770 2024-03-08T06:05:09.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.279+0000 7f0755c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f075006d520 0x7f075015b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:09.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.279+0000 7f0755c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f075006d520 0x7f075015b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33460/0 (socket says 172.21.15.40:33460) 2024-03-08T06:05:09.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.279+0000 7f0755c00640 1 -- 172.21.15.40:0/3001220876 learned_addr learned my addr 172.21.15.40:0/3001220876 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:09.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.279+0000 7f0755200640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0750071a60 0x7f07501e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:09.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.280+0000 7f0756600640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0750144770 0x7f07501e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:09.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.280+0000 7f0755c00640 1 -- 172.21.15.40:0/3001220876 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0750144770 msgr2=0x7f07501e96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:09.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.280+0000 7f0755c00640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0750144770 0x7f07501e96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.280+0000 7f0755c00640 1 -- 172.21.15.40:0/3001220876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0750071a60 msgr2=0x7f07501e9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:09.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.280+0000 7f0755c00640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0750071a60 0x7f07501e9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.280+0000 7f0755c00640 1 -- 172.21.15.40:0/3001220876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07501e6960 con 0x7f075006d520 2024-03-08T06:05:09.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.280+0000 7f0755c00640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f075006d520 0x7f075015b760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f074400c3f0 tx=0x7f074400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:09.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.281+0000 7f074d600640 1 -- 172.21.15.40:0/3001220876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07440052d0 con 0x7f075006d520 2024-03-08T06:05:09.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.281+0000 7f074d600640 1 -- 172.21.15.40:0/3001220876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0744005490 con 0x7f075006d520 2024-03-08T06:05:09.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.281+0000 7f0757000640 1 -- 172.21.15.40:0/3001220876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07501e6c40 con 0x7f075006d520 2024-03-08T06:05:09.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.281+0000 7f074d600640 1 -- 172.21.15.40:0/3001220876 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744007260 con 0x7f075006d520 2024-03-08T06:05:09.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.281+0000 7f0757000640 1 -- 172.21.15.40:0/3001220876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07501e54a0 con 0x7f075006d520 2024-03-08T06:05:09.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.282+0000 7f0757000640 1 -- 172.21.15.40:0/3001220876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0750075090 con 0x7f075006d520 2024-03-08T06:05:09.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.284+0000 7f074d600640 1 -- 172.21.15.40:0/3001220876 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0744007400 con 0x7f075006d520 2024-03-08T06:05:09.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.284+0000 7f074d600640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f073006b4d0 0x7f073006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:09.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.285+0000 7f074d600640 1 -- 172.21.15.40:0/3001220876 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f07440892f0 con 0x7f075006d520 2024-03-08T06:05:09.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.285+0000 7f0755200640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f073006b4d0 0x7f073006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:09.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.286+0000 7f0755200640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f073006b4d0 0x7f073006d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07501e6750 tx=0x7f073c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:09.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.289+0000 7f074d600640 1 -- 172.21.15.40:0/3001220876 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0744057e60 con 0x7f075006d520 2024-03-08T06:05:09.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.491+0000 7f0757000640 1 -- 172.21.15.40:0/3001220876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f075006f580 con 0x7f075006d520 2024-03-08T06:05:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.940+0000 7f074d600640 1 -- 172.21.15.40:0/3001220876 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f074405ce20 con 0x7f075006d520 2024-03-08T06:05:09.941 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-03-08T06:05:09.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.944+0000 7f0757000640 1 -- 172.21.15.40:0/3001220876 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f073006b4d0 msgr2=0x7f073006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:09.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.944+0000 7f0757000640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f073006b4d0 0x7f073006d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07501e6750 tx=0x7f073c009190 comp rx=0 tx=0).stop 2024-03-08T06:05:09.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.944+0000 7f0757000640 1 -- 172.21.15.40:0/3001220876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f075006d520 msgr2=0x7f075015b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:09.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.944+0000 7f0757000640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f075006d520 0x7f075015b760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f074400c3f0 tx=0x7f074400c8b0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.945+0000 7f0757000640 1 -- 172.21.15.40:0/3001220876 shutdown_connections 2024-03-08T06:05:09.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.945+0000 7f0757000640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f073006b4d0 0x7f073006d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.945+0000 7f0757000640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0750144770 0x7f07501e96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.945+0000 7f0757000640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0750071a60 0x7f07501e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.945+0000 7f0757000640 1 --2- 172.21.15.40:0/3001220876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f075006d520 0x7f075015b760 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:09.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.945+0000 7f0757000640 1 -- 172.21.15.40:0/3001220876 >> 172.21.15.40:0/3001220876 conn(0x7f075006ac50 msgr2=0x7f07500ae2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:09.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.946+0000 7f0757000640 1 -- 172.21.15.40:0/3001220876 shutdown_connections 2024-03-08T06:05:09.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:09.946+0000 7f0757000640 1 -- 172.21.15.40:0/3001220876 wait complete. 2024-03-08T06:05:09.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-08T06:05:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.193+0000 7f885be00640 1 Processor -- start 2024-03-08T06:05:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.193+0000 7f885be00640 1 -- start start 2024-03-08T06:05:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.195+0000 7f885be00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f885c0ce8a0 0x7f885c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.195+0000 7f885be00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f885c0cb370 0x7f885c0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.195+0000 7f885be00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885c15eed0 con 0x7f885c15a790 2024-03-08T06:05:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.195+0000 7f885be00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885c15f1b0 con 0x7f885c0cb370 2024-03-08T06:05:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.195+0000 7f885be00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0cd020 con 0x7f885c0ce8a0 2024-03-08T06:05:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.195+0000 7f885aa00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f885c15a790 0x7f885c15e6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43408/0 (socket says 172.21.15.40:43408) 2024-03-08T06:05:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.196+0000 7f885aa00640 1 -- 172.21.15.40:0/3236849851 learned_addr learned my addr 172.21.15.40:0/3236849851 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.196+0000 7f885b400640 1 --2- 172.21.15.40:0/3236849851 >> v2:172.21.15.40:3300/0 conn(0x7f885c0cb370 0x7f885c0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.196+0000 7f885a000640 1 --2- 172.21.15.40:0/3236849851 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f885c0ce8a0 0x7f885c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.196+0000 7f8859600640 1 -- 172.21.15.40:0/3236849851 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969876466 0 0) 0x7f885c15eed0 con 0x7f885c15a790 2024-03-08T06:05:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.196+0000 7f8859600640 1 -- 172.21.15.40:0/3236849851 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8840003520 con 0x7f885c15a790 2024-03-08T06:05:10.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.196+0000 7f885b400640 1 -- 172.21.15.40:0/3236849851 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f885c0ce8a0 msgr2=0x7f885c15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:10.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.196+0000 7f885b400640 1 --2- 172.21.15.40:0/3236849851 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f885c0ce8a0 0x7f885c15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:10.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.196+0000 7f885b400640 1 -- 172.21.15.40:0/3236849851 >> v1:172.21.15.40:6789/0 conn(0x7f885c15a790 legacy=0x7f885c15e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:10.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.196+0000 7f885b400640 1 -- 172.21.15.40:0/3236849851 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c0cd300 con 0x7f885c0cb370 2024-03-08T06:05:10.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.198+0000 7f885b400640 1 --2- 172.21.15.40:0/3236849851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f885c0cb370 0x7f885c0cb7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8854009e10 tx=0x7f885402e2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d61c026a3c28f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:10.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.198+0000 7f8859600640 1 -- 172.21.15.40:0/3236849851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885403b070 con 0x7f885c0cb370 2024-03-08T06:05:10.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.198+0000 7f8859600640 1 -- 172.21.15.40:0/3236849851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885402ee50 con 0x7f885c0cb370 2024-03-08T06:05:10.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.198+0000 7f8859600640 1 -- 172.21.15.40:0/3236849851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8854036860 con 0x7f885c0cb370 2024-03-08T06:05:10.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.199+0000 7f885be00640 1 -- 172.21.15.40:0/3236849851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f885c0cb370 msgr2=0x7f885c0cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:10.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.199+0000 7f885be00640 1 --2- 172.21.15.40:0/3236849851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f885c0cb370 0x7f885c0cb7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8854009e10 tx=0x7f885402e2a0 comp rx=0 tx=0).stop 2024-03-08T06:05:10.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.199+0000 7f885be00640 1 -- 172.21.15.40:0/3236849851 shutdown_connections 2024-03-08T06:05:10.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.199+0000 7f885be00640 1 --2- 172.21.15.40:0/3236849851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f885c0cb370 0x7f885c0cb7a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:10.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.199+0000 7f885be00640 1 --2- 172.21.15.40:0/3236849851 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f885c0ce8a0 0x7f885c15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:10.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.199+0000 7f885be00640 1 -- 172.21.15.40:0/3236849851 >> 172.21.15.40:0/3236849851 conn(0x7f885c099a70 msgr2=0x7f885c099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:10.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.199+0000 7f885be00640 1 -- 172.21.15.40:0/3236849851 shutdown_connections 2024-03-08T06:05:10.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.199+0000 7f885be00640 1 -- 172.21.15.40:0/3236849851 wait complete. 2024-03-08T06:05:10.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.201+0000 7f885be00640 1 Processor -- start 2024-03-08T06:05:10.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.201+0000 7f885be00640 1 -- start start 2024-03-08T06:05:10.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.202+0000 7f885be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0cb370 0x7f885c14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:10.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.202+0000 7f885be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f885c0ce8a0 0x7f885c0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:10.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.202+0000 7f885be00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f885c15a790 0x7f885c0c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:10.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.202+0000 7f885be00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0cd9a0 con 0x7f885c0ce8a0 2024-03-08T06:05:10.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.202+0000 7f885be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0cd820 con 0x7f885c0cb370 2024-03-08T06:05:10.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.202+0000 7f885be00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0cdb20 con 0x7f885c15a790 2024-03-08T06:05:10.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.202+0000 7f885a000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f885c0ce8a0 0x7f885c0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:10.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.202+0000 7f885a000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f885c0ce8a0 0x7f885c0b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33468/0 (socket says 172.21.15.40:33468) 2024-03-08T06:05:10.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.202+0000 7f885a000640 1 -- 172.21.15.40:0/93958867 learned_addr learned my addr 172.21.15.40:0/93958867 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:10.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.203+0000 7f885b400640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f885c15a790 0x7f885c0c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:10.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.203+0000 7f885aa00640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0cb370 0x7f885c14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:10.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.203+0000 7f885a000640 1 -- 172.21.15.40:0/93958867 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f885c15a790 msgr2=0x7f885c0c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:10.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.203+0000 7f885a000640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f885c15a790 0x7f885c0c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:10.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.203+0000 7f885a000640 1 -- 172.21.15.40:0/93958867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0cb370 msgr2=0x7f885c14ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:10.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.203+0000 7f885a000640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0cb370 0x7f885c14ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:10.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.203+0000 7f885a000640 1 -- 172.21.15.40:0/93958867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c0b4050 con 0x7f885c0ce8a0 2024-03-08T06:05:10.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.203+0000 7f885a000640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f885c0ce8a0 0x7f885c0b3750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f884800e9e0 tx=0x7f884800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:10.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.203+0000 7f8852a00640 1 -- 172.21.15.40:0/93958867 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848017070 con 0x7f885c0ce8a0 2024-03-08T06:05:10.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.204+0000 7f8852a00640 1 -- 172.21.15.40:0/93958867 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884800c9d0 con 0x7f885c0ce8a0 2024-03-08T06:05:10.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.204+0000 7f885be00640 1 -- 172.21.15.40:0/93958867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885c0c8b50 con 0x7f885c0ce8a0 2024-03-08T06:05:10.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.204+0000 7f8852a00640 1 -- 172.21.15.40:0/93958867 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884800cdc0 con 0x7f885c0ce8a0 2024-03-08T06:05:10.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.204+0000 7f885be00640 1 -- 172.21.15.40:0/93958867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885c0c9110 con 0x7f885c0ce8a0 2024-03-08T06:05:10.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.205+0000 7f885be00640 1 -- 172.21.15.40:0/93958867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885c075090 con 0x7f885c0ce8a0 2024-03-08T06:05:10.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.206+0000 7f8852a00640 1 -- 172.21.15.40:0/93958867 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f884801a070 con 0x7f885c0ce8a0 2024-03-08T06:05:10.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.207+0000 7f8852a00640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f883c06b4d0 0x7f883c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:10.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.207+0000 7f8852a00640 1 -- 172.21.15.40:0/93958867 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f884808a470 con 0x7f885c0ce8a0 2024-03-08T06:05:10.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.207+0000 7f885aa00640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f883c06b4d0 0x7f883c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:10.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.208+0000 7f885aa00640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f883c06b4d0 0x7f883c06d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f884c007fd0 tx=0x7f884c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:10.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.212+0000 7f8852a00640 1 -- 172.21.15.40:0/93958867 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88480594c0 con 0x7f885c0ce8a0 2024-03-08T06:05:10.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.412+0000 7f885be00640 1 -- 172.21.15.40:0/93958867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f885c06f2c0 con 0x7f885c0ce8a0 2024-03-08T06:05:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.958+0000 7f8852a00640 1 -- 172.21.15.40:0/93958867 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8848058dc0 con 0x7f885c0ce8a0 2024-03-08T06:05:10.959 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-03-08T06:05:10.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.964+0000 7f885be00640 1 -- 172.21.15.40:0/93958867 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f883c06b4d0 msgr2=0x7f883c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:10.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.964+0000 7f885be00640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f883c06b4d0 0x7f883c06d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f884c007fd0 tx=0x7f884c009040 comp rx=0 tx=0).stop 2024-03-08T06:05:10.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.964+0000 7f885be00640 1 -- 172.21.15.40:0/93958867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f885c0ce8a0 msgr2=0x7f885c0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:10.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.964+0000 7f885be00640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f885c0ce8a0 0x7f885c0b3750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f884800e9e0 tx=0x7f884800eea0 comp rx=0 tx=0).stop 2024-03-08T06:05:10.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.964+0000 7f885be00640 1 -- 172.21.15.40:0/93958867 shutdown_connections 2024-03-08T06:05:10.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.964+0000 7f885be00640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f883c06b4d0 0x7f883c06d990 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f884c007fd0 tx=0x7f884c009040 comp rx=0 tx=0).stop 2024-03-08T06:05:10.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.964+0000 7f885be00640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f885c15a790 0x7f885c0c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:10.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.964+0000 7f885be00640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f885c0ce8a0 0x7f885c0b3750 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:10.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.964+0000 7f885be00640 1 --2- 172.21.15.40:0/93958867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f885c0cb370 0x7f885c14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:10.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.964+0000 7f885be00640 1 -- 172.21.15.40:0/93958867 >> 172.21.15.40:0/93958867 conn(0x7f885c099a70 msgr2=0x7f885c15b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:10.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.966+0000 7f885be00640 1 -- 172.21.15.40:0/93958867 shutdown_connections 2024-03-08T06:05:10.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:10.967+0000 7f885be00640 1 -- 172.21.15.40:0/93958867 wait complete. 2024-03-08T06:05:11.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-08T06:05:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.190+0000 7efe58e00640 1 Processor -- start 2024-03-08T06:05:11.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.190+0000 7efe58e00640 1 -- start start 2024-03-08T06:05:11.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.192+0000 7efe58e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe541455b0 0x7efe5415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.192+0000 7efe58e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efe541604f0 0x7efe540cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.192+0000 7efe58e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe540cd5a0 con 0x7efe54071950 2024-03-08T06:05:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.192+0000 7efe58e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe540cdc20 con 0x7efe541604f0 2024-03-08T06:05:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.192+0000 7efe58e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe540ccfe0 con 0x7efe541455b0 2024-03-08T06:05:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.192+0000 7efe52800640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efe54071950 0x7efe540ce0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43420/0 (socket says 172.21.15.40:43420) 2024-03-08T06:05:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.192+0000 7efe52800640 1 -- 172.21.15.40:0/260649672 learned_addr learned my addr 172.21.15.40:0/260649672 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.192+0000 7efe51e00640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe541455b0 0x7efe5415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe4be00640 1 --2- 172.21.15.40:0/260649672 >> v2:172.21.15.53:3300/0 conn(0x7efe541604f0 0x7efe540cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139191804 0 0) 0x7efe540cd5a0 con 0x7efe54071950 2024-03-08T06:05:11.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe38003520 con 0x7efe54071950 2024-03-08T06:05:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1958718695 0 0) 0x7efe38003520 con 0x7efe54071950 2024-03-08T06:05:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe541455b0 msgr2=0x7efe5415c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe541455b0 0x7efe5415c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 >> v2:172.21.15.53:3300/0 conn(0x7efe541604f0 msgr2=0x7efe540cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 --2- 172.21.15.40:0/260649672 >> v2:172.21.15.53:3300/0 conn(0x7efe541604f0 0x7efe540cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe540cd2c0 con 0x7efe54071950 2024-03-08T06:05:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7efe4c0019f0 con 0x7efe54071950 2024-03-08T06:05:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 >> v1:172.21.15.40:6789/0 conn(0x7efe54071950 legacy=0x7efe540ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe380039b0 0x7efe38003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe38004800 0x7efe38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe541455b0 0x7efe380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe540cdc20 con 0x7efe38004800 2024-03-08T06:05:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe540ccfe0 con 0x7efe380039b0 2024-03-08T06:05:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3800b9c0 con 0x7efe541455b0 2024-03-08T06:05:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.193+0000 7efe4be00640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe38004800 0x7efe38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.194+0000 7efe52800640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe541455b0 0x7efe380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.194+0000 7efe4be00640 1 -- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe541455b0 msgr2=0x7efe380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.194+0000 7efe51e00640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe380039b0 0x7efe38003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.194+0000 7efe4be00640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe541455b0 0x7efe380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.194+0000 7efe4be00640 1 -- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe380039b0 msgr2=0x7efe38003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.194+0000 7efe4be00640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe380039b0 0x7efe38003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.194+0000 7efe4be00640 1 -- 172.21.15.40:0/260649672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe38003520 con 0x7efe38004800 2024-03-08T06:05:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.194+0000 7efe52800640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe541455b0 0x7efe380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.195+0000 7efe4be00640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe38004800 0x7efe38004c80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009ac0 tx=0x7efe38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8e585d4e5ec80a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.195+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c039070 con 0x7efe38004800 2024-03-08T06:05:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.195+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c02c6b0 con 0x7efe38004800 2024-03-08T06:05:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.195+0000 7efe51400640 1 -- 172.21.15.40:0/260649672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c009690 con 0x7efe38004800 2024-03-08T06:05:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.196+0000 7efe58e00640 1 -- 172.21.15.40:0/260649672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe38004800 msgr2=0x7efe38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.196+0000 7efe58e00640 1 --2- 172.21.15.40:0/260649672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe38004800 0x7efe38004c80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009ac0 tx=0x7efe38005830 comp rx=0 tx=0).stop 2024-03-08T06:05:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.196+0000 7efe52800640 1 -- 172.21.15.40:0/260649672 reap_dead start 2024-03-08T06:05:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.196+0000 7efe58e00640 1 -- 172.21.15.40:0/260649672 shutdown_connections 2024-03-08T06:05:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.196+0000 7efe58e00640 1 -- 172.21.15.40:0/260649672 >> 172.21.15.40:0/260649672 conn(0x7efe5406ae30 msgr2=0x7efe5406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.196+0000 7efe58e00640 1 -- 172.21.15.40:0/260649672 shutdown_connections 2024-03-08T06:05:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.196+0000 7efe58e00640 1 -- 172.21.15.40:0/260649672 wait complete. 2024-03-08T06:05:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.198+0000 7efe58e00640 1 Processor -- start 2024-03-08T06:05:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.198+0000 7efe58e00640 1 -- start start 2024-03-08T06:05:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.199+0000 7efe58e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe38004800 0x7efe5413d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.199+0000 7efe58e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54140f00 0x7efe54141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.199+0000 7efe58e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe54152d10 0x7efe54153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.199+0000 7efe58e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe540ccbd0 con 0x7efe54140f00 2024-03-08T06:05:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.199+0000 7efe58e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe54145990 con 0x7efe54152d10 2024-03-08T06:05:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.199+0000 7efe58e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe540ccd50 con 0x7efe38004800 2024-03-08T06:05:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.199+0000 7efe4be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54140f00 0x7efe54141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.199+0000 7efe4be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54140f00 0x7efe54141380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33490/0 (socket says 172.21.15.40:33490) 2024-03-08T06:05:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.199+0000 7efe4be00640 1 -- 172.21.15.40:0/2912322119 learned_addr learned my addr 172.21.15.40:0/2912322119 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.200+0000 7efe51e00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe38004800 0x7efe5413d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.200+0000 7efe4be00640 1 -- 172.21.15.40:0/2912322119 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe38004800 msgr2=0x7efe5413d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.200+0000 7efe4be00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe38004800 0x7efe5413d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.200+0000 7efe52800640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe54152d10 0x7efe54153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.200+0000 7efe4be00640 1 -- 172.21.15.40:0/2912322119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe54152d10 msgr2=0x7efe54153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.200+0000 7efe4be00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe54152d10 0x7efe54153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.200+0000 7efe4be00640 1 -- 172.21.15.40:0/2912322119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5413d640 con 0x7efe54140f00 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.200+0000 7efe52800640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe54152d10 0x7efe54153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.200+0000 7efe4be00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54140f00 0x7efe54141380 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efe3c000c00 tx=0x7efe3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.201+0000 7efe4a000640 1 -- 172.21.15.40:0/2912322119 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c039050 con 0x7efe54140f00 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.201+0000 7efe4a000640 1 -- 172.21.15.40:0/2912322119 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c045070 con 0x7efe54140f00 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.201+0000 7efe58e00640 1 -- 172.21.15.40:0/2912322119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe541536d0 con 0x7efe54140f00 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.201+0000 7efe4a000640 1 -- 172.21.15.40:0/2912322119 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0044e0 con 0x7efe54140f00 2024-03-08T06:05:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.201+0000 7efe58e00640 1 -- 172.21.15.40:0/2912322119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5413c340 con 0x7efe54140f00 2024-03-08T06:05:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.202+0000 7efe58e00640 1 -- 172.21.15.40:0/2912322119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe54067cb0 con 0x7efe54140f00 2024-03-08T06:05:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.203+0000 7efe4a000640 1 -- 172.21.15.40:0/2912322119 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe3c003a50 con 0x7efe54140f00 2024-03-08T06:05:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.204+0000 7efe4a000640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe3006b500 0x7efe3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.204+0000 7efe4a000640 1 -- 172.21.15.40:0/2912322119 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7efe3c0ac070 con 0x7efe54140f00 2024-03-08T06:05:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.204+0000 7efe51e00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe3006b500 0x7efe3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.205+0000 7efe51e00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe3006b500 0x7efe3006d9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe44009830 tx=0x7efe44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.209+0000 7efe4a000640 1 -- 172.21.15.40:0/2912322119 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe3c04f020 con 0x7efe54140f00 2024-03-08T06:05:11.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.443+0000 7efe58e00640 1 -- 172.21.15.40:0/2912322119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efe540729c0 con 0x7efe54140f00 2024-03-08T06:05:11.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.979+0000 7efe4a000640 1 -- 172.21.15.40:0/2912322119 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7efe5413d640 con 0x7efe54140f00 2024-03-08T06:05:11.984 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 4 and data pool 5 2024-03-08T06:05:11.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.988+0000 7efe58e00640 1 -- 172.21.15.40:0/2912322119 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe3006b500 msgr2=0x7efe3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:11.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.988+0000 7efe58e00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe3006b500 0x7efe3006d9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe44009830 tx=0x7efe44008040 comp rx=0 tx=0).stop 2024-03-08T06:05:11.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.989+0000 7efe58e00640 1 -- 172.21.15.40:0/2912322119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54140f00 msgr2=0x7efe54141380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:11.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.989+0000 7efe58e00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54140f00 0x7efe54141380 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efe3c000c00 tx=0x7efe3c004730 comp rx=0 tx=0).stop 2024-03-08T06:05:11.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.991+0000 7efe58e00640 1 -- 172.21.15.40:0/2912322119 shutdown_connections 2024-03-08T06:05:11.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.991+0000 7efe58e00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7efe3006b500 0x7efe3006d9c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:11.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.991+0000 7efe58e00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe54152d10 0x7efe54153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:11.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.991+0000 7efe58e00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe54140f00 0x7efe54141380 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:11.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.991+0000 7efe58e00640 1 --2- 172.21.15.40:0/2912322119 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe38004800 0x7efe5413d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:11.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.991+0000 7efe58e00640 1 -- 172.21.15.40:0/2912322119 >> 172.21.15.40:0/2912322119 conn(0x7efe5406ae30 msgr2=0x7efe541448e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:11.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.992+0000 7efe58e00640 1 -- 172.21.15.40:0/2912322119 shutdown_connections 2024-03-08T06:05:11.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:11.993+0000 7efe58e00640 1 -- 172.21.15.40:0/2912322119 wait complete. 2024-03-08T06:05:12.022 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:05:12.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.200+0000 7f82b8400640 1 Processor -- start 2024-03-08T06:05:12.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.200+0000 7f82b8400640 1 -- start start 2024-03-08T06:05:12.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.201+0000 7f82b8400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82b0071ab0 0x7f82b015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.202+0000 7f82b8400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b015d0c0 con 0x7f82b006d600 2024-03-08T06:05:12.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.202+0000 7f82b8400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b006da80 con 0x7f82b0071e90 2024-03-08T06:05:12.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.202+0000 7f82b8400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b006ef40 con 0x7f82b0071ab0 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.202+0000 7f82b5000640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f82b006d600 0x7f82b0144d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43424/0 (socket says 172.21.15.40:43424) 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.202+0000 7f82b5000640 1 -- 172.21.15.40:0/2659809330 learned_addr learned my addr 172.21.15.40:0/2659809330 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.202+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836775160 0 0) 0x7f829c001110 con 0x7f82b006d600 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.202+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8298003520 con 0x7f82b006d600 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.202+0000 7f82b5a00640 1 --2- 172.21.15.40:0/2659809330 >> v2:172.21.15.53:3300/0 conn(0x7f82b0071ab0 0x7f82b015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1764100776 0 0) 0x7f8298003520 con 0x7f82b006d600 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 >> v2:172.21.15.53:3300/0 conn(0x7f82b0071ab0 msgr2=0x7f82b015cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 --2- 172.21.15.40:0/2659809330 >> v2:172.21.15.53:3300/0 conn(0x7f82b0071ab0 0x7f82b015cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 >> v1:172.21.15.53:6789/0 conn(0x7f82b0071e90 legacy=0x7f82b015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b006dd60 con 0x7f82b006d600 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f829c001840 con 0x7f82b006d600 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 >> v1:172.21.15.40:6789/0 conn(0x7f82b006d600 legacy=0x7f82b0144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 --2- 172.21.15.40:0/2659809330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82980039b0 0x7f8298003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 --2- 172.21.15.40:0/2659809330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 0x7f8298004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 --2- 172.21.15.40:0/2659809330 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82980051c0 0x7f829800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829800c0c0 con 0x7f8298004800 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829800c270 con 0x7f82980039b0 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829800c470 con 0x7f82980051c0 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82b5000640 1 --2- 172.21.15.40:0/2659809330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 0x7f8298004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.203+0000 7f82b5a00640 1 --2- 172.21.15.40:0/2659809330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82980039b0 0x7f8298003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.204+0000 7f82b5000640 1 -- 172.21.15.40:0/2659809330 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82980051c0 msgr2=0x7f829800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.204+0000 7f82b5000640 1 --2- 172.21.15.40:0/2659809330 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82980051c0 0x7f829800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.204+0000 7f82b5000640 1 -- 172.21.15.40:0/2659809330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82980039b0 msgr2=0x7f8298003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.204+0000 7f82b5000640 1 --2- 172.21.15.40:0/2659809330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82980039b0 0x7f8298003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.204+0000 7f82b5000640 1 -- 172.21.15.40:0/2659809330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8298003520 con 0x7f8298004800 2024-03-08T06:05:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.204+0000 7f82b6400640 1 -- 172.21.15.40:0/2659809330 reap_dead start 2024-03-08T06:05:12.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.205+0000 7f82b5000640 1 --2- 172.21.15.40:0/2659809330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 0x7f8298004c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f82b015c140 tx=0x7f8298005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a47ad6426ac086 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:12.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.205+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c00c0e0 con 0x7f8298004800 2024-03-08T06:05:12.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.205+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c030070 con 0x7f8298004800 2024-03-08T06:05:12.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.205+0000 7f82afe00640 1 -- 172.21.15.40:0/2659809330 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c02ce30 con 0x7f8298004800 2024-03-08T06:05:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.205+0000 7f82b8400640 1 -- 172.21.15.40:0/2659809330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 msgr2=0x7f8298004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.205+0000 7f82b8400640 1 --2- 172.21.15.40:0/2659809330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 0x7f8298004c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f82b015c140 tx=0x7f8298005830 comp rx=0 tx=0).stop 2024-03-08T06:05:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.205+0000 7f82b8400640 1 -- 172.21.15.40:0/2659809330 shutdown_connections 2024-03-08T06:05:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.205+0000 7f82b8400640 1 --2- 172.21.15.40:0/2659809330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 0x7f8298004c80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.205+0000 7f82b8400640 1 -- 172.21.15.40:0/2659809330 >> 172.21.15.40:0/2659809330 conn(0x7f82b006ae30 msgr2=0x7f82b006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.206+0000 7f82b8400640 1 -- 172.21.15.40:0/2659809330 shutdown_connections 2024-03-08T06:05:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.206+0000 7f82b8400640 1 -- 172.21.15.40:0/2659809330 wait complete. 2024-03-08T06:05:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.207+0000 7f82b8400640 1 Processor -- start 2024-03-08T06:05:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.207+0000 7f82b8400640 1 -- start start 2024-03-08T06:05:12.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.208+0000 7f82b8400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 0x7f82b015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.208+0000 7f82b8400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82b00b0760 0x7f82b00b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.208+0000 7f82b8400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82b00c48e0 0x7f82b00c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.208+0000 7f82b8400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b006e600 con 0x7f8298004800 2024-03-08T06:05:12.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.208+0000 7f82b8400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b006e320 con 0x7f82b00b0760 2024-03-08T06:05:12.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.208+0000 7f82b8400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b015cb80 con 0x7f82b00c48e0 2024-03-08T06:05:12.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.209+0000 7f82b5a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 0x7f82b015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.209+0000 7f82b5a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 0x7f82b015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33498/0 (socket says 172.21.15.40:33498) 2024-03-08T06:05:12.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.209+0000 7f82b5a00640 1 -- 172.21.15.40:0/940438131 learned_addr learned my addr 172.21.15.40:0/940438131 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:12.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.209+0000 7f82b5a00640 1 -- 172.21.15.40:0/940438131 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82b00c48e0 msgr2=0x7f82b00c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:12.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.209+0000 7f82b6400640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82b00c48e0 0x7f82b00c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.209+0000 7f82b5000640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82b00b0760 0x7f82b00b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.209+0000 7f82b5a00640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82b00c48e0 0x7f82b00c4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.209+0000 7f82b5a00640 1 -- 172.21.15.40:0/940438131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82b00b0760 msgr2=0x7f82b00b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:12.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.209+0000 7f82b5a00640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82b00b0760 0x7f82b00b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.209+0000 7f82b5a00640 1 -- 172.21.15.40:0/940438131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00c55d0 con 0x7f8298004800 2024-03-08T06:05:12.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.209+0000 7f82b5a00640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 0x7f82b015a730 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f82a400f600 tx=0x7f82a400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:12.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.210+0000 7f82ad600640 1 -- 172.21.15.40:0/940438131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4019020 con 0x7f8298004800 2024-03-08T06:05:12.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.210+0000 7f82ad600640 1 -- 172.21.15.40:0/940438131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a4014870 con 0x7f8298004800 2024-03-08T06:05:12.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.210+0000 7f82b8400640 1 -- 172.21.15.40:0/940438131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b00c58b0 con 0x7f8298004800 2024-03-08T06:05:12.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.210+0000 7f82ad600640 1 -- 172.21.15.40:0/940438131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4014c20 con 0x7f8298004800 2024-03-08T06:05:12.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.210+0000 7f82b8400640 1 -- 172.21.15.40:0/940438131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b01365b0 con 0x7f8298004800 2024-03-08T06:05:12.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.211+0000 7f82b8400640 1 -- 172.21.15.40:0/940438131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b00b1120 con 0x7f8298004800 2024-03-08T06:05:12.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.212+0000 7f82ad600640 1 -- 172.21.15.40:0/940438131 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82a401c020 con 0x7f8298004800 2024-03-08T06:05:12.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.213+0000 7f82ad600640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f829006b4d0 0x7f829006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.213+0000 7f82ad600640 1 -- 172.21.15.40:0/940438131 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82a408ab30 con 0x7f8298004800 2024-03-08T06:05:12.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.213+0000 7f82b5000640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f829006b4d0 0x7f829006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.214+0000 7f82b5000640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f829006b4d0 0x7f829006d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f829c034000 tx=0x7f829c007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:12.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.217+0000 7f82ad600640 1 -- 172.21.15.40:0/940438131 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82a4023020 con 0x7f8298004800 2024-03-08T06:05:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.413+0000 7f82b8400640 1 -- 172.21.15.40:0/940438131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82b00adc60 con 0x7f8298004800 2024-03-08T06:05:12.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.416+0000 7f82ad600640 1 -- 172.21.15.40:0/940438131 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16598 (secure 0 0 0) 0x7f82a40599a0 con 0x7f8298004800 2024-03-08T06:05:12.416 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:05:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:05:11.445280+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:05:09.544904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T06:05:10.468274+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:05:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.418+0000 7f82b8400640 1 -- 172.21.15.40:0/940438131 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f829006b4d0 msgr2=0x7f829006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.418+0000 7f82b8400640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f829006b4d0 0x7f829006d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f829c034000 tx=0x7f829c007000 comp rx=0 tx=0).stop 2024-03-08T06:05:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.418+0000 7f82b8400640 1 -- 172.21.15.40:0/940438131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 msgr2=0x7f82b015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.418+0000 7f82b8400640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 0x7f82b015a730 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f82a400f600 tx=0x7f82a400fac0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.418+0000 7f82b8400640 1 -- 172.21.15.40:0/940438131 shutdown_connections 2024-03-08T06:05:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.418+0000 7f82b8400640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f829006b4d0 0x7f829006d990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.418+0000 7f82b8400640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82b00c48e0 0x7f82b00c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.418+0000 7f82b8400640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82b00b0760 0x7f82b00b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.418+0000 7f82b8400640 1 --2- 172.21.15.40:0/940438131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8298004800 0x7f82b015a730 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.418+0000 7f82b8400640 1 -- 172.21.15.40:0/940438131 >> 172.21.15.40:0/940438131 conn(0x7f82b006ae30 msgr2=0x7f82b00affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.419+0000 7f82b8400640 1 -- 172.21.15.40:0/940438131 shutdown_connections 2024-03-08T06:05:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.419+0000 7f82b8400640 1 -- 172.21.15.40:0/940438131 wait complete. 2024-03-08T06:05:12.435 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:05:12.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.646+0000 7f9f35200640 1 Processor -- start 2024-03-08T06:05:12.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.647+0000 7f9f35200640 1 -- start start 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.648+0000 7f9f35200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9f301614f0 0x7f9f300ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.648+0000 7f9f35200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f300d1130 0x7f9f300cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.648+0000 7f9f35200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f300d2b90 con 0x7f9f300d3630 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.648+0000 7f9f35200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f300b6d50 con 0x7f9f301614f0 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.648+0000 7f9f35200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300d3370 con 0x7f9f300d1130 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.648+0000 7f9f2de00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9f301614f0 0x7f9f300ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.648+0000 7f9f2de00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9f301614f0 0x7f9f300ce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33508/0 (socket says 172.21.15.40:33508) 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.648+0000 7f9f2de00640 1 -- 172.21.15.40:0/1283499407 learned_addr learned my addr 172.21.15.40:0/1283499407 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.649+0000 7f9f2e800640 1 --2- 172.21.15.40:0/1283499407 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f300d1130 0x7f9f300cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.649+0000 7f9f2de00640 1 -- 172.21.15.40:0/1283499407 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f300d1130 msgr2=0x7f9f300cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.649+0000 7f9f2de00640 1 --2- 172.21.15.40:0/1283499407 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f300d1130 0x7f9f300cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.649+0000 7f9f2de00640 1 -- 172.21.15.40:0/1283499407 >> v1:172.21.15.53:6789/0 conn(0x7f9f300d3630 legacy=0x7f9f30161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:12.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.649+0000 7f9f2de00640 1 -- 172.21.15.40:0/1283499407 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f300b9820 con 0x7f9f301614f0 2024-03-08T06:05:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.650+0000 7f9f2de00640 1 --2- 172.21.15.40:0/1283499407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f301614f0 0x7f9f300ce910 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009870 tx=0x7f9f2002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfa71743e73c787 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.650+0000 7f9f2ca00640 1 -- 172.21.15.40:0/1283499407 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f200317c0 con 0x7f9f301614f0 2024-03-08T06:05:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.650+0000 7f9f2ca00640 1 -- 172.21.15.40:0/1283499407 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f20037070 con 0x7f9f301614f0 2024-03-08T06:05:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.650+0000 7f9f2ca00640 1 -- 172.21.15.40:0/1283499407 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20031b80 con 0x7f9f301614f0 2024-03-08T06:05:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.651+0000 7f9f35200640 1 -- 172.21.15.40:0/1283499407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f301614f0 msgr2=0x7f9f300ce910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.651+0000 7f9f35200640 1 --2- 172.21.15.40:0/1283499407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f301614f0 0x7f9f300ce910 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009870 tx=0x7f9f2002b860 comp rx=0 tx=0).stop 2024-03-08T06:05:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.651+0000 7f9f35200640 1 -- 172.21.15.40:0/1283499407 shutdown_connections 2024-03-08T06:05:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.651+0000 7f9f35200640 1 --2- 172.21.15.40:0/1283499407 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f300d1130 0x7f9f300cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.651+0000 7f9f35200640 1 --2- 172.21.15.40:0/1283499407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f301614f0 0x7f9f300ce910 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.651+0000 7f9f35200640 1 -- 172.21.15.40:0/1283499407 >> 172.21.15.40:0/1283499407 conn(0x7f9f3006c8f0 msgr2=0x7f9f3006cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.651+0000 7f9f35200640 1 -- 172.21.15.40:0/1283499407 shutdown_connections 2024-03-08T06:05:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.651+0000 7f9f35200640 1 -- 172.21.15.40:0/1283499407 wait complete. 2024-03-08T06:05:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.653+0000 7f9f35200640 1 Processor -- start 2024-03-08T06:05:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.653+0000 7f9f35200640 1 -- start start 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.654+0000 7f9f35200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f300d1130 0x7f9f300cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.654+0000 7f9f35200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f300d3630 0x7f9f3015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.654+0000 7f9f35200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f301614f0 0x7f9f30160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.654+0000 7f9f35200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300d2ee0 con 0x7f9f300d1130 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.654+0000 7f9f35200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300b95d0 con 0x7f9f301614f0 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.654+0000 7f9f35200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f301618d0 con 0x7f9f300d3630 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.654+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f300d1130 0x7f9f300cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.654+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f300d1130 0x7f9f300cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33510/0 (socket says 172.21.15.40:33510) 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.654+0000 7f9f2de00640 1 -- 172.21.15.40:0/398019938 learned_addr learned my addr 172.21.15.40:0/398019938 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.654+0000 7f9f2e800640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f301614f0 0x7f9f30160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.655+0000 7f9f2d400640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f300d3630 0x7f9f3015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.655+0000 7f9f2de00640 1 -- 172.21.15.40:0/398019938 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f300d3630 msgr2=0x7f9f3015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.655+0000 7f9f2de00640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f300d3630 0x7f9f3015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.655+0000 7f9f2de00640 1 -- 172.21.15.40:0/398019938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f301614f0 msgr2=0x7f9f30160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.655+0000 7f9f2de00640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f301614f0 0x7f9f30160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.655+0000 7f9f2de00640 1 -- 172.21.15.40:0/398019938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f301607d0 con 0x7f9f300d1130 2024-03-08T06:05:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.655+0000 7f9f2de00640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f300d1130 0x7f9f300cd690 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f2002b8a0 tx=0x7f9f200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.655+0000 7f9f26000640 1 -- 172.21.15.40:0/398019938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20032650 con 0x7f9f300d1130 2024-03-08T06:05:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.655+0000 7f9f26000640 1 -- 172.21.15.40:0/398019938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f20032810 con 0x7f9f300d1130 2024-03-08T06:05:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.655+0000 7f9f26000640 1 -- 172.21.15.40:0/398019938 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20032c00 con 0x7f9f300d1130 2024-03-08T06:05:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.655+0000 7f9f35200640 1 -- 172.21.15.40:0/398019938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f30160ab0 con 0x7f9f300d1130 2024-03-08T06:05:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.656+0000 7f9f35200640 1 -- 172.21.15.40:0/398019938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f300c8bb0 con 0x7f9f300d1130 2024-03-08T06:05:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.657+0000 7f9f35200640 1 -- 172.21.15.40:0/398019938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f30067cb0 con 0x7f9f300d1130 2024-03-08T06:05:12.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.658+0000 7f9f26000640 1 -- 172.21.15.40:0/398019938 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f2004c060 con 0x7f9f300d1130 2024-03-08T06:05:12.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.658+0000 7f9f26000640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f0c06b4d0 0x7f9f0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.659+0000 7f9f26000640 1 -- 172.21.15.40:0/398019938 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f200abdd0 con 0x7f9f300d1130 2024-03-08T06:05:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.659+0000 7f9f2d400640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f0c06b4d0 0x7f9f0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.659+0000 7f9f2d400640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f0c06b4d0 0x7f9f0c06d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9f300d2b20 tx=0x7f9f18009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:12.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.663+0000 7f9f26000640 1 -- 172.21.15.40:0/398019938 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f20040070 con 0x7f9f300d1130 2024-03-08T06:05:12.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.854+0000 7f9f35200640 1 -- 172.21.15.40:0/398019938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f300b64e0 con 0x7f9f300d1130 2024-03-08T06:05:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.858+0000 7f9f26000640 1 -- 172.21.15.40:0/398019938 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16598 (secure 0 0 0) 0x7f9f2007abb0 con 0x7f9f300d1130 2024-03-08T06:05:12.858 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:05:12.858 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:05:11.445280+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:05:09.544904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T06:05:10.468274+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:05:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 -- 172.21.15.40:0/398019938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f0c06b4d0 msgr2=0x7f9f0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f0c06b4d0 0x7f9f0c06d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9f300d2b20 tx=0x7f9f18009040 comp rx=0 tx=0).stop 2024-03-08T06:05:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 -- 172.21.15.40:0/398019938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f300d1130 msgr2=0x7f9f300cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f300d1130 0x7f9f300cd690 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f2002b8a0 tx=0x7f9f200046c0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 -- 172.21.15.40:0/398019938 shutdown_connections 2024-03-08T06:05:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f9f0c06b4d0 0x7f9f0c06d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f301614f0 0x7f9f30160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f300d3630 0x7f9f3015fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 --2- 172.21.15.40:0/398019938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f300d1130 0x7f9f300cd690 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 -- 172.21.15.40:0/398019938 >> 172.21.15.40:0/398019938 conn(0x7f9f3006c8f0 msgr2=0x7f9f300b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 -- 172.21.15.40:0/398019938 shutdown_connections 2024-03-08T06:05:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:12.861+0000 7f9f35200640 1 -- 172.21.15.40:0/398019938 wait complete. 2024-03-08T06:05:12.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-08T06:05:13.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.100+0000 7f10dce00640 1 Processor -- start 2024-03-08T06:05:13.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.100+0000 7f10dce00640 1 -- start start 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.102+0000 7f10dce00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f10d815de40 0x7f10d8161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.102+0000 7f10dce00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10d80b9610 0x7f10d80ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.102+0000 7f10dce00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d815e220 con 0x7f10d80cee40 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.102+0000 7f10dce00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d80cf270 con 0x7f10d80b9610 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.102+0000 7f10dce00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d8162010 con 0x7f10d815de40 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.102+0000 7f10d6a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10d80b9610 0x7f10d80ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.102+0000 7f10d7e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f10d80cee40 0x7f10d80b55b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43436/0 (socket says 172.21.15.40:43436) 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.102+0000 7f10d6a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10d80b9610 0x7f10d80ce900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33524/0 (socket says 172.21.15.40:33524) 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.102+0000 7f10d7e00640 1 -- 172.21.15.40:0/2680812774 learned_addr learned my addr 172.21.15.40:0/2680812774 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.103+0000 7f10d7400640 1 --2- 172.21.15.40:0/2680812774 >> v2:172.21.15.53:3300/0 conn(0x7f10d815de40 0x7f10d8161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.103+0000 7f10d6a00640 1 -- 172.21.15.40:0/2680812774 >> v2:172.21.15.53:3300/0 conn(0x7f10d815de40 msgr2=0x7f10d8161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.103+0000 7f10d6a00640 1 --2- 172.21.15.40:0/2680812774 >> v2:172.21.15.53:3300/0 conn(0x7f10d815de40 0x7f10d8161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.103+0000 7f10d6a00640 1 -- 172.21.15.40:0/2680812774 >> v1:172.21.15.40:6789/0 conn(0x7f10d80cee40 legacy=0x7f10d80b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:13.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.103+0000 7f10d6a00640 1 -- 172.21.15.40:0/2680812774 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d81622f0 con 0x7f10d80b9610 2024-03-08T06:05:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.103+0000 7f10d6000640 1 -- 172.21.15.40:0/2680812774 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609709830 0 0) 0x7f10d815e220 con 0x7f10d80cee40 2024-03-08T06:05:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.104+0000 7f10d6a00640 1 --2- 172.21.15.40:0/2680812774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10d80b9610 0x7f10d80ce900 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f10d80b6720 tx=0x7f10c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324cbeb4cfc67637 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.104+0000 7f10d6000640 1 -- 172.21.15.40:0/2680812774 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0039070 con 0x7f10d80b9610 2024-03-08T06:05:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.104+0000 7f10d6000640 1 -- 172.21.15.40:0/2680812774 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c002cc80 con 0x7f10d80b9610 2024-03-08T06:05:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.104+0000 7f10d6000640 1 -- 172.21.15.40:0/2680812774 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0034660 con 0x7f10d80b9610 2024-03-08T06:05:13.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.105+0000 7f10dce00640 1 -- 172.21.15.40:0/2680812774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10d80b9610 msgr2=0x7f10d80ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:13.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.105+0000 7f10dce00640 1 --2- 172.21.15.40:0/2680812774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10d80b9610 0x7f10d80ce900 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f10d80b6720 tx=0x7f10c002bed0 comp rx=0 tx=0).stop 2024-03-08T06:05:13.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.105+0000 7f10dce00640 1 -- 172.21.15.40:0/2680812774 shutdown_connections 2024-03-08T06:05:13.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.105+0000 7f10dce00640 1 --2- 172.21.15.40:0/2680812774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10d80b9610 0x7f10d80ce900 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:13.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.105+0000 7f10dce00640 1 --2- 172.21.15.40:0/2680812774 >> v2:172.21.15.53:3300/0 conn(0x7f10d815de40 0x7f10d8161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:13.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.105+0000 7f10dce00640 1 -- 172.21.15.40:0/2680812774 >> 172.21.15.40:0/2680812774 conn(0x7f10d806c500 msgr2=0x7f10d806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:13.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.105+0000 7f10dce00640 1 -- 172.21.15.40:0/2680812774 shutdown_connections 2024-03-08T06:05:13.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.105+0000 7f10dce00640 1 -- 172.21.15.40:0/2680812774 wait complete. 2024-03-08T06:05:13.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.107+0000 7f10dce00640 1 Processor -- start 2024-03-08T06:05:13.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.107+0000 7f10dce00640 1 -- start start 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.109+0000 7f10dce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d80b9610 0x7f10d80c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.109+0000 7f10dce00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10d80cee40 0x7f10d81e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.109+0000 7f10dce00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f10d815de40 0x7f10d80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.109+0000 7f10dce00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80b9a90 con 0x7f10d80cee40 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.109+0000 7f10dce00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80d20a0 con 0x7f10d80b9610 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.109+0000 7f10dce00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80d35d0 con 0x7f10d815de40 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.109+0000 7f10d6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10d80cee40 0x7f10d81e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.109+0000 7f10d6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10d80cee40 0x7f10d81e98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33540/0 (socket says 172.21.15.40:33540) 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.109+0000 7f10d6a00640 1 -- 172.21.15.40:0/3785572030 learned_addr learned my addr 172.21.15.40:0/3785572030 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.109+0000 7f10d7400640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d80b9610 0x7f10d80c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.109+0000 7f10d7e00640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f10d815de40 0x7f10d80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10d6a00640 1 -- 172.21.15.40:0/3785572030 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f10d815de40 msgr2=0x7f10d80b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:13.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10d6a00640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f10d815de40 0x7f10d80b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:13.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10d6a00640 1 -- 172.21.15.40:0/3785572030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d80b9610 msgr2=0x7f10d80c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:13.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10d6a00640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d80b9610 0x7f10d80c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:13.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10d6a00640 1 -- 172.21.15.40:0/3785572030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d80c8390 con 0x7f10d80cee40 2024-03-08T06:05:13.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10d7400640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d80b9610 0x7f10d80c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:13.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10d7e00640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f10d815de40 0x7f10d80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:13.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10d6a00640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10d80cee40 0x7f10d81e98b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f10c0037040 tx=0x7f10c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:13.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10cf400640 1 -- 172.21.15.40:0/3785572030 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0039050 con 0x7f10d80cee40 2024-03-08T06:05:13.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10cf400640 1 -- 172.21.15.40:0/3785572030 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c0045070 con 0x7f10d80cee40 2024-03-08T06:05:13.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10cf400640 1 -- 172.21.15.40:0/3785572030 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c003e850 con 0x7f10d80cee40 2024-03-08T06:05:13.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10dce00640 1 -- 172.21.15.40:0/3785572030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d80c8670 con 0x7f10d80cee40 2024-03-08T06:05:13.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.110+0000 7f10dce00640 1 -- 172.21.15.40:0/3785572030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d80c8c30 con 0x7f10d80cee40 2024-03-08T06:05:13.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.112+0000 7f10dce00640 1 -- 172.21.15.40:0/3785572030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d8075090 con 0x7f10d80cee40 2024-03-08T06:05:13.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.112+0000 7f10cf400640 1 -- 172.21.15.40:0/3785572030 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f10c003e9f0 con 0x7f10d80cee40 2024-03-08T06:05:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.114+0000 7f10cf400640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f10b406b500 0x7f10b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.114+0000 7f10cf400640 1 -- 172.21.15.40:0/3785572030 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10c00acf70 con 0x7f10d80cee40 2024-03-08T06:05:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.114+0000 7f10d7400640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f10b406b500 0x7f10b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.115+0000 7f10d7400640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f10b406b500 0x7f10b406d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f10c8006fd0 tx=0x7f10c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:13.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.118+0000 7f10cf400640 1 -- 172.21.15.40:0/3785572030 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10c007bf00 con 0x7f10d80cee40 2024-03-08T06:05:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:13.349+0000 7f10dce00640 1 -- 172.21.15.40:0/3785572030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f10d8161220 con 0x7f10d80cee40 2024-03-08T06:05:13.911 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-03-08T06:05:13.911+0000 7fc769600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-08T06:05:14.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.000+0000 7f10cf400640 1 -- 172.21.15.40:0/3785572030 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f10c007b800 con 0x7f10d80cee40 2024-03-08T06:05:14.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.003+0000 7f10dce00640 1 -- 172.21.15.40:0/3785572030 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f10b406b500 msgr2=0x7f10b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:14.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.003+0000 7f10dce00640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f10b406b500 0x7f10b406d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f10c8006fd0 tx=0x7f10c8008040 comp rx=0 tx=0).stop 2024-03-08T06:05:14.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.003+0000 7f10dce00640 1 -- 172.21.15.40:0/3785572030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10d80cee40 msgr2=0x7f10d81e98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:14.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.003+0000 7f10dce00640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10d80cee40 0x7f10d81e98b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f10c0037040 tx=0x7f10c0004a40 comp rx=0 tx=0).stop 2024-03-08T06:05:14.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.003+0000 7f10dce00640 1 -- 172.21.15.40:0/3785572030 shutdown_connections 2024-03-08T06:05:14.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.003+0000 7f10dce00640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f10b406b500 0x7f10b406d9c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:14.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.004+0000 7f10dce00640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f10d815de40 0x7f10d80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:14.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.004+0000 7f10dce00640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10d80cee40 0x7f10d81e98b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:14.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.004+0000 7f10dce00640 1 --2- 172.21.15.40:0/3785572030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d80b9610 0x7f10d80c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:14.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.004+0000 7f10dce00640 1 -- 172.21.15.40:0/3785572030 >> 172.21.15.40:0/3785572030 conn(0x7f10d806c500 msgr2=0x7f10d8161860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:14.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.004+0000 7f10dce00640 1 -- 172.21.15.40:0/3785572030 shutdown_connections 2024-03-08T06:05:14.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.004+0000 7f10dce00640 1 -- 172.21.15.40:0/3785572030 wait complete. 2024-03-08T06:05:14.023 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-08T06:05:14.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-08T06:05:14.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.239+0000 7f6a3f200640 1 Processor -- start 2024-03-08T06:05:14.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.240+0000 7f6a3f200640 1 -- start start 2024-03-08T06:05:14.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.241+0000 7f6a3f200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a40147040 0x7f6a4015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:14.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.241+0000 7f6a3f200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a400b2970 0x7f6a4006e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:14.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.241+0000 7f6a3f200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a4015e840 con 0x7f6a400716e0 2024-03-08T06:05:14.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.241+0000 7f6a3f200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4015eb20 con 0x7f6a40147040 2024-03-08T06:05:14.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.241+0000 7f6a3f200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4015ee00 con 0x7f6a400b2970 2024-03-08T06:05:14.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.241+0000 7f6a3de00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a40147040 0x7f6a4015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:14.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.241+0000 7f6a3de00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a40147040 0x7f6a4015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33556/0 (socket says 172.21.15.40:33556) 2024-03-08T06:05:14.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.241+0000 7f6a3de00640 1 -- 172.21.15.40:0/3273888697 learned_addr learned my addr 172.21.15.40:0/3273888697 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:14.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.242+0000 7f6a3d400640 1 --2- 172.21.15.40:0/3273888697 >> v2:172.21.15.53:3300/0 conn(0x7f6a400b2970 0x7f6a4006e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:14.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.242+0000 7f6a3de00640 1 -- 172.21.15.40:0/3273888697 >> v2:172.21.15.53:3300/0 conn(0x7f6a400b2970 msgr2=0x7f6a4006e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:14.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.242+0000 7f6a3de00640 1 --2- 172.21.15.40:0/3273888697 >> v2:172.21.15.53:3300/0 conn(0x7f6a400b2970 0x7f6a4006e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:14.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.242+0000 7f6a3de00640 1 -- 172.21.15.40:0/3273888697 >> v1:172.21.15.53:6789/0 conn(0x7f6a400716e0 legacy=0x7f6a400ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:14.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.242+0000 7f6a3de00640 1 -- 172.21.15.40:0/3273888697 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a400ae0d0 con 0x7f6a40147040 2024-03-08T06:05:14.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.243+0000 7f6a3de00640 1 --2- 172.21.15.40:0/3273888697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a40147040 0x7f6a4015e1a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009870 tx=0x7f6a3002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50da344507d3c47a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:14.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.243+0000 7f6a3ca00640 1 -- 172.21.15.40:0/3273888697 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30035020 con 0x7f6a40147040 2024-03-08T06:05:14.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.243+0000 7f6a3ca00640 1 -- 172.21.15.40:0/3273888697 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a30031870 con 0x7f6a40147040 2024-03-08T06:05:14.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.244+0000 7f6a3ca00640 1 -- 172.21.15.40:0/3273888697 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30031c40 con 0x7f6a40147040 2024-03-08T06:05:14.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.244+0000 7f6a3f200640 1 -- 172.21.15.40:0/3273888697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a40147040 msgr2=0x7f6a4015e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:14.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.244+0000 7f6a3f200640 1 --2- 172.21.15.40:0/3273888697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a40147040 0x7f6a4015e1a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009870 tx=0x7f6a3002b910 comp rx=0 tx=0).stop 2024-03-08T06:05:14.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.244+0000 7f6a3f200640 1 -- 172.21.15.40:0/3273888697 shutdown_connections 2024-03-08T06:05:14.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.244+0000 7f6a3f200640 1 --2- 172.21.15.40:0/3273888697 >> v2:172.21.15.53:3300/0 conn(0x7f6a400b2970 0x7f6a4006e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:14.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.244+0000 7f6a3f200640 1 --2- 172.21.15.40:0/3273888697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a40147040 0x7f6a4015e1a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:14.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.244+0000 7f6a3f200640 1 -- 172.21.15.40:0/3273888697 >> 172.21.15.40:0/3273888697 conn(0x7f6a400b1850 msgr2=0x7f6a400b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:14.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.244+0000 7f6a3f200640 1 -- 172.21.15.40:0/3273888697 shutdown_connections 2024-03-08T06:05:14.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.244+0000 7f6a3f200640 1 -- 172.21.15.40:0/3273888697 wait complete. 2024-03-08T06:05:14.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.246+0000 7f6a3f200640 1 Processor -- start 2024-03-08T06:05:14.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.246+0000 7f6a3f200640 1 -- start start 2024-03-08T06:05:14.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3f200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a400716e0 0x7f6a40141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:14.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3f200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6a400b2970 0x7f6a401436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:14.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a40147040 0x7f6a400b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:14.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3f200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40071b30 con 0x7f6a400716e0 2024-03-08T06:05:14.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4006edc0 con 0x7f6a40147040 2024-03-08T06:05:14.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3f200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40071cb0 con 0x7f6a400b2970 2024-03-08T06:05:14.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a400716e0 0x7f6a40141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:14.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a400716e0 0x7f6a40141340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33572/0 (socket says 172.21.15.40:33572) 2024-03-08T06:05:14.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3de00640 1 -- 172.21.15.40:0/2611666262 learned_addr learned my addr 172.21.15.40:0/2611666262 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3de00640 1 -- 172.21.15.40:0/2611666262 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6a400b2970 msgr2=0x7f6a401436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3de00640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6a400b2970 0x7f6a401436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3de00640 1 -- 172.21.15.40:0/2611666262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a40147040 msgr2=0x7f6a400b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.247+0000 7f6a3e800640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a40147040 0x7f6a400b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.248+0000 7f6a3de00640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a40147040 0x7f6a400b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.248+0000 7f6a3de00640 1 -- 172.21.15.40:0/2611666262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4012e8b0 con 0x7f6a400716e0 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.248+0000 7f6a3de00640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a400716e0 0x7f6a40141340 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009870 tx=0x7f6a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.248+0000 7f6a36000640 1 -- 172.21.15.40:0/2611666262 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30040070 con 0x7f6a400716e0 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.248+0000 7f6a36000640 1 -- 172.21.15.40:0/2611666262 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3003a420 con 0x7f6a400716e0 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.248+0000 7f6a36000640 1 -- 172.21.15.40:0/2611666262 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30035040 con 0x7f6a400716e0 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.248+0000 7f6a3f200640 1 -- 172.21.15.40:0/2611666262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a40143c10 con 0x7f6a400716e0 2024-03-08T06:05:14.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.248+0000 7f6a3f200640 1 -- 172.21.15.40:0/2611666262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4013ed90 con 0x7f6a400716e0 2024-03-08T06:05:14.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.249+0000 7f6a3f200640 1 -- 172.21.15.40:0/2611666262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a40075090 con 0x7f6a400716e0 2024-03-08T06:05:14.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.250+0000 7f6a36000640 1 -- 172.21.15.40:0/2611666262 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a3004b060 con 0x7f6a400716e0 2024-03-08T06:05:14.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.251+0000 7f6a36000640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6a2006b4d0 0x7f6a2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:14.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.251+0000 7f6a36000640 1 -- 172.21.15.40:0/2611666262 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a300ac190 con 0x7f6a400716e0 2024-03-08T06:05:14.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.251+0000 7f6a3d400640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6a2006b4d0 0x7f6a2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:14.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.252+0000 7f6a3d400640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6a2006b4d0 0x7f6a2006d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c006fd0 tx=0x7f6a2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:14.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.256+0000 7f6a36000640 1 -- 172.21.15.40:0/2611666262 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a4012e8b0 con 0x7f6a400716e0 2024-03-08T06:05:14.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:14.484+0000 7f6a3f200640 1 -- 172.21.15.40:0/2611666262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6a4013eb20 con 0x7f6a400716e0 2024-03-08T06:05:15.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.003+0000 7f6a36000640 1 -- 172.21.15.40:0/2611666262 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6a3007af80 con 0x7f6a400716e0 2024-03-08T06:05:15.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.007+0000 7f6a3f200640 1 -- 172.21.15.40:0/2611666262 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6a2006b4d0 msgr2=0x7f6a2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.007+0000 7f6a3f200640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6a2006b4d0 0x7f6a2006d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c006fd0 tx=0x7f6a2c008040 comp rx=0 tx=0).stop 2024-03-08T06:05:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.007+0000 7f6a3f200640 1 -- 172.21.15.40:0/2611666262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a400716e0 msgr2=0x7f6a40141340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.007+0000 7f6a3f200640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a400716e0 0x7f6a40141340 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009870 tx=0x7f6a300046c0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.007+0000 7f6a3f200640 1 -- 172.21.15.40:0/2611666262 shutdown_connections 2024-03-08T06:05:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.007+0000 7f6a3f200640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6a2006b4d0 0x7f6a2006d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.007+0000 7f6a3f200640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a40147040 0x7f6a400b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.007+0000 7f6a3f200640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6a400b2970 0x7f6a401436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.007+0000 7f6a3f200640 1 --2- 172.21.15.40:0/2611666262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a400716e0 0x7f6a40141340 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.007+0000 7f6a3f200640 1 -- 172.21.15.40:0/2611666262 >> 172.21.15.40:0/2611666262 conn(0x7f6a400b1850 msgr2=0x7f6a40144a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.008+0000 7f6a3f200640 1 -- 172.21.15.40:0/2611666262 shutdown_connections 2024-03-08T06:05:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.008+0000 7f6a3f200640 1 -- 172.21.15.40:0/2611666262 wait complete. 2024-03-08T06:05:15.028 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-08T06:05:15.028 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:05:15.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.249+0000 7ff8e6a00640 1 Processor -- start 2024-03-08T06:05:15.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.249+0000 7ff8e6a00640 1 -- start start 2024-03-08T06:05:15.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.251+0000 7ff8e6a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff8e015df20 0x7ff8e00ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.251+0000 7ff8e6a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8e00b6bd0 0x7ff8e0162400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.251+0000 7ff8e6a00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e015e300 con 0x7ff8e00d0aa0 2024-03-08T06:05:15.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.251+0000 7ff8e6a00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e00d3990 con 0x7ff8e015df20 2024-03-08T06:05:15.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.251+0000 7ff8e6a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0162940 con 0x7ff8e00b6bd0 2024-03-08T06:05:15.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.252+0000 7ff8e4c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8e00b6bd0 0x7ff8e0162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.252+0000 7ff8e5600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff8e015df20 0x7ff8e00ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.252+0000 7ff8e5600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff8e015df20 0x7ff8e00ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:36120/0 (socket says 172.21.15.40:36120) 2024-03-08T06:05:15.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.252+0000 7ff8e5600640 1 -- 172.21.15.40:0/4069495269 learned_addr learned my addr 172.21.15.40:0/4069495269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:15.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.253+0000 7ff8e4c00640 1 -- 172.21.15.40:0/4069495269 >> v2:172.21.15.53:3300/0 conn(0x7ff8e015df20 msgr2=0x7ff8e00ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:15.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.253+0000 7ff8e4c00640 1 --2- 172.21.15.40:0/4069495269 >> v2:172.21.15.53:3300/0 conn(0x7ff8e015df20 0x7ff8e00ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.253+0000 7ff8e4c00640 1 -- 172.21.15.40:0/4069495269 >> v1:172.21.15.53:6789/0 conn(0x7ff8e00d0aa0 legacy=0x7ff8e00b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.253+0000 7ff8e4c00640 1 -- 172.21.15.40:0/4069495269 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e00cf030 con 0x7ff8e00b6bd0 2024-03-08T06:05:15.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.253+0000 7ff8e5600640 1 --2- 172.21.15.40:0/4069495269 >> v2:172.21.15.53:3300/0 conn(0x7ff8e015df20 0x7ff8e00ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:05:15.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.254+0000 7ff8e4c00640 1 --2- 172.21.15.40:0/4069495269 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8e00b6bd0 0x7ff8e0162400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0097a0 tx=0x7ff8cc02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=620bfc9ff9be9311 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:15.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.255+0000 7ff8dfe00640 1 -- 172.21.15.40:0/4069495269 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc034070 con 0x7ff8e00b6bd0 2024-03-08T06:05:15.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.255+0000 7ff8dfe00640 1 -- 172.21.15.40:0/4069495269 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc030800 con 0x7ff8e00b6bd0 2024-03-08T06:05:15.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.255+0000 7ff8dfe00640 1 -- 172.21.15.40:0/4069495269 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc030bd0 con 0x7ff8e00b6bd0 2024-03-08T06:05:15.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.255+0000 7ff8e6a00640 1 -- 172.21.15.40:0/4069495269 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8e00b6bd0 msgr2=0x7ff8e0162400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.255+0000 7ff8e6a00640 1 --2- 172.21.15.40:0/4069495269 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8e00b6bd0 0x7ff8e0162400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0097a0 tx=0x7ff8cc02b940 comp rx=0 tx=0).stop 2024-03-08T06:05:15.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.256+0000 7ff8e6a00640 1 -- 172.21.15.40:0/4069495269 shutdown_connections 2024-03-08T06:05:15.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.256+0000 7ff8e6a00640 1 --2- 172.21.15.40:0/4069495269 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8e00b6bd0 0x7ff8e0162400 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.256+0000 7ff8e6a00640 1 --2- 172.21.15.40:0/4069495269 >> v2:172.21.15.53:3300/0 conn(0x7ff8e015df20 0x7ff8e00ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.256+0000 7ff8e6a00640 1 -- 172.21.15.40:0/4069495269 >> 172.21.15.40:0/4069495269 conn(0x7ff8e006c740 msgr2=0x7ff8e006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:15.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.256+0000 7ff8e6a00640 1 -- 172.21.15.40:0/4069495269 shutdown_connections 2024-03-08T06:05:15.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.256+0000 7ff8e6a00640 1 -- 172.21.15.40:0/4069495269 wait complete. 2024-03-08T06:05:15.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.258+0000 7ff8e6a00640 1 Processor -- start 2024-03-08T06:05:15.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.258+0000 7ff8e6a00640 1 -- start start 2024-03-08T06:05:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.259+0000 7ff8e6a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8e00b6bd0 0x7ff8e01f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.259+0000 7ff8e6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e00d0aa0 0x7ff8e00c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.259+0000 7ff8e6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e015df20 0x7ff8e00c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.259+0000 7ff8e6a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e00d2180 con 0x7ff8e015df20 2024-03-08T06:05:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.259+0000 7ff8e6a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e00d1ea0 con 0x7ff8e00d0aa0 2024-03-08T06:05:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.259+0000 7ff8e6a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e00d2460 con 0x7ff8e00b6bd0 2024-03-08T06:05:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.260+0000 7ff8e6000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e015df20 0x7ff8e00c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.260+0000 7ff8e6000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e015df20 0x7ff8e00c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33586/0 (socket says 172.21.15.40:33586) 2024-03-08T06:05:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.260+0000 7ff8e6000640 1 -- 172.21.15.40:0/1271846920 learned_addr learned my addr 172.21.15.40:0/1271846920 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:15.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.260+0000 7ff8e4c00640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e00d0aa0 0x7ff8e00c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.260+0000 7ff8e5600640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8e00b6bd0 0x7ff8e01f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.260+0000 7ff8e6000640 1 -- 172.21.15.40:0/1271846920 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8e00b6bd0 msgr2=0x7ff8e01f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.260+0000 7ff8e6000640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8e00b6bd0 0x7ff8e01f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.260+0000 7ff8e6000640 1 -- 172.21.15.40:0/1271846920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e00d0aa0 msgr2=0x7ff8e00c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.260+0000 7ff8e6000640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e00d0aa0 0x7ff8e00c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.260+0000 7ff8e6000640 1 -- 172.21.15.40:0/1271846920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e01f7190 con 0x7ff8e015df20 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.261+0000 7ff8e6000640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e015df20 0x7ff8e00c9d50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c7a0 tx=0x7ff8d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.261+0000 7ff8dd600640 1 -- 172.21.15.40:0/1271846920 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8003b70 con 0x7ff8e015df20 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.261+0000 7ff8dd600640 1 -- 172.21.15.40:0/1271846920 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8003d30 con 0x7ff8e015df20 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.261+0000 7ff8dd600640 1 -- 172.21.15.40:0/1271846920 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800aa90 con 0x7ff8e015df20 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.261+0000 7ff8e6a00640 1 -- 172.21.15.40:0/1271846920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e01f7470 con 0x7ff8e015df20 2024-03-08T06:05:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.261+0000 7ff8e6a00640 1 -- 172.21.15.40:0/1271846920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e01f7a30 con 0x7ff8e015df20 2024-03-08T06:05:15.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.263+0000 7ff8e6a00640 1 -- 172.21.15.40:0/1271846920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e01f7c30 con 0x7ff8e015df20 2024-03-08T06:05:15.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.264+0000 7ff8dd600640 1 -- 172.21.15.40:0/1271846920 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8d800ac30 con 0x7ff8e015df20 2024-03-08T06:05:15.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.264+0000 7ff8dd600640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff8c006b4d0 0x7ff8c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.265+0000 7ff8dd600640 1 -- 172.21.15.40:0/1271846920 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8d808a270 con 0x7ff8e015df20 2024-03-08T06:05:15.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.265+0000 7ff8e5600640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff8c006b4d0 0x7ff8c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.266+0000 7ff8e5600640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff8c006b4d0 0x7ff8c006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0160120 tx=0x7ff8d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:15.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.269+0000 7ff8dd600640 1 -- 172.21.15.40:0/1271846920 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8d8058fd0 con 0x7ff8e015df20 2024-03-08T06:05:15.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.512+0000 7ff8e6a00640 1 -- 172.21.15.40:0/1271846920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8e0075100 con 0x7ff8e015df20 2024-03-08T06:05:15.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.513+0000 7ff8dd600640 1 -- 172.21.15.40:0/1271846920 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4935 (secure 0 0 0) 0x7ff8d80588d0 con 0x7ff8e015df20 2024-03-08T06:05:15.513 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:05:15.513 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/2194146146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6835","nonce":2194146146},{"type":"v1","addr":"172.21.15.40:6837","nonce":2194146146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/831964898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":831964898},{"type":"v1","addr":"172.21.15.53:6835","nonce":831964898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4723,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2951334634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2951334634},{"type":"v1","addr":"172.21.15.53:6833","nonce":2951334634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:05:11.446139+0000","modified":"2024-03-08T06:05:14.485839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6836/1742457959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1742457959},{"type":"v1","addr":"172.21.15.40:6836","nonce":1742457959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:05:15.513 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2024-03-08T06:05:15.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 -- 172.21.15.40:0/1271846920 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff8c006b4d0 msgr2=0x7ff8c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff8c006b4d0 0x7ff8c006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0160120 tx=0x7ff8d4008040 comp rx=0 tx=0).stop 2024-03-08T06:05:15.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 -- 172.21.15.40:0/1271846920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e015df20 msgr2=0x7ff8e00c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e015df20 0x7ff8e00c9d50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c7a0 tx=0x7ff8d800cc60 comp rx=0 tx=0).stop 2024-03-08T06:05:15.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 -- 172.21.15.40:0/1271846920 shutdown_connections 2024-03-08T06:05:15.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff8c006b4d0 0x7ff8c006d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e015df20 0x7ff8e00c9d50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e00d0aa0 0x7ff8e00c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 --2- 172.21.15.40:0/1271846920 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8e00b6bd0 0x7ff8e01f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 -- 172.21.15.40:0/1271846920 >> 172.21.15.40:0/1271846920 conn(0x7ff8e006c740 msgr2=0x7ff8e00b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:15.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 -- 172.21.15.40:0/1271846920 shutdown_connections 2024-03-08T06:05:15.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.516+0000 7ff8e6a00640 1 -- 172.21.15.40:0/1271846920 wait complete. 2024-03-08T06:05:15.536 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:05:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.757+0000 7fbcfaa00640 1 Processor -- start 2024-03-08T06:05:15.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.757+0000 7fbcfaa00640 1 -- start start 2024-03-08T06:05:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.759+0000 7fbcfaa00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcf406d600 0x7fbcf4144d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.759+0000 7fbcfaa00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbcf4071e90 0x7fbcf41452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.759+0000 7fbcfaa00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf406da80 con 0x7fbcf4071ab0 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.759+0000 7fbcfaa00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf406ef40 con 0x7fbcf406d600 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.759+0000 7fbcfaa00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf41457e0 con 0x7fbcf4071e90 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.759+0000 7fbcf3400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcf406d600 0x7fbcf4144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.759+0000 7fbcf3e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbcf4071ab0 0x7fbcf415cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43440/0 (socket says 172.21.15.40:43440) 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.759+0000 7fbcf3400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbcf406d600 0x7fbcf4144d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33592/0 (socket says 172.21.15.40:33592) 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.759+0000 7fbcf3e00640 1 -- 172.21.15.40:0/2430078766 learned_addr learned my addr 172.21.15.40:0/2430078766 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.760+0000 7fbcf8a00640 1 --2- 172.21.15.40:0/2430078766 >> v2:172.21.15.53:3300/0 conn(0x7fbcf4071e90 0x7fbcf41452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.760+0000 7fbcf3400640 1 -- 172.21.15.40:0/2430078766 >> v2:172.21.15.53:3300/0 conn(0x7fbcf4071e90 msgr2=0x7fbcf41452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.760+0000 7fbcf3400640 1 --2- 172.21.15.40:0/2430078766 >> v2:172.21.15.53:3300/0 conn(0x7fbcf4071e90 0x7fbcf41452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.760+0000 7fbcf3400640 1 -- 172.21.15.40:0/2430078766 >> v1:172.21.15.40:6789/0 conn(0x7fbcf4071ab0 legacy=0x7fbcf415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.760+0000 7fbcf3400640 1 -- 172.21.15.40:0/2430078766 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf406dd60 con 0x7fbcf406d600 2024-03-08T06:05:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.761+0000 7fbcf3400640 1 --2- 172.21.15.40:0/2430078766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf406d600 0x7fbcf4144d60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40cf2e0 tx=0x7fbce002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30918994037935e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.761+0000 7fbcf2a00640 1 -- 172.21.15.40:0/2430078766 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0035020 con 0x7fbcf406d600 2024-03-08T06:05:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.761+0000 7fbcf2a00640 1 -- 172.21.15.40:0/2430078766 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce0031830 con 0x7fbcf406d600 2024-03-08T06:05:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.761+0000 7fbcf2a00640 1 -- 172.21.15.40:0/2430078766 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0031c00 con 0x7fbcf406d600 2024-03-08T06:05:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.762+0000 7fbcfaa00640 1 -- 172.21.15.40:0/2430078766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf406d600 msgr2=0x7fbcf4144d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.762+0000 7fbcfaa00640 1 --2- 172.21.15.40:0/2430078766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf406d600 0x7fbcf4144d60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40cf2e0 tx=0x7fbce002b8d0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.762+0000 7fbcfaa00640 1 -- 172.21.15.40:0/2430078766 shutdown_connections 2024-03-08T06:05:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.762+0000 7fbcfaa00640 1 --2- 172.21.15.40:0/2430078766 >> v2:172.21.15.53:3300/0 conn(0x7fbcf4071e90 0x7fbcf41452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.762+0000 7fbcfaa00640 1 --2- 172.21.15.40:0/2430078766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf406d600 0x7fbcf4144d60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.762+0000 7fbcfaa00640 1 -- 172.21.15.40:0/2430078766 >> 172.21.15.40:0/2430078766 conn(0x7fbcf406ae30 msgr2=0x7fbcf406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.762+0000 7fbcfaa00640 1 -- 172.21.15.40:0/2430078766 shutdown_connections 2024-03-08T06:05:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.762+0000 7fbcfaa00640 1 -- 172.21.15.40:0/2430078766 wait complete. 2024-03-08T06:05:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.764+0000 7fbcfaa00640 1 Processor -- start 2024-03-08T06:05:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.764+0000 7fbcfaa00640 1 -- start start 2024-03-08T06:05:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.765+0000 7fbcfaa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf406d600 0x7fbcf40ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.765+0000 7fbcfaa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcf4071ab0 0x7fbcf40cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.765+0000 7fbcfaa00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbcf4071e90 0x7fbcf41f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.765+0000 7fbcfaa00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf406e600 con 0x7fbcf406d600 2024-03-08T06:05:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.765+0000 7fbcfaa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf406e100 con 0x7fbcf4071ab0 2024-03-08T06:05:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.765+0000 7fbcfaa00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf415d3a0 con 0x7fbcf4071e90 2024-03-08T06:05:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.765+0000 7fbcf3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf406d600 0x7fbcf40ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.765+0000 7fbcf3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf406d600 0x7fbcf40ace30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33594/0 (socket says 172.21.15.40:33594) 2024-03-08T06:05:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.765+0000 7fbcf3e00640 1 -- 172.21.15.40:0/3454568738 learned_addr learned my addr 172.21.15.40:0/3454568738 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.765+0000 7fbcf3400640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcf4071ab0 0x7fbcf40cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.765+0000 7fbcf3e00640 1 -- 172.21.15.40:0/3454568738 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbcf4071e90 msgr2=0x7fbcf41f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcf8a00640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbcf4071e90 0x7fbcf41f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcf3e00640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbcf4071e90 0x7fbcf41f7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcf3e00640 1 -- 172.21.15.40:0/3454568738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcf4071ab0 msgr2=0x7fbcf40cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcf3e00640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcf4071ab0 0x7fbcf40cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcf3e00640 1 -- 172.21.15.40:0/3454568738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf40ad660 con 0x7fbcf406d600 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcf3400640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcf4071ab0 0x7fbcf40cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcf3e00640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf406d600 0x7fbcf40ace30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbce400c400 tx=0x7fbce400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcebe00640 1 -- 172.21.15.40:0/3454568738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4017070 con 0x7fbcf406d600 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcebe00640 1 -- 172.21.15.40:0/3454568738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce4005ce0 con 0x7fbcf406d600 2024-03-08T06:05:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcebe00640 1 -- 172.21.15.40:0/3454568738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce40072f0 con 0x7fbcf406d600 2024-03-08T06:05:15.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcfaa00640 1 -- 172.21.15.40:0/3454568738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf40ad940 con 0x7fbcf406d600 2024-03-08T06:05:15.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.766+0000 7fbcfaa00640 1 -- 172.21.15.40:0/3454568738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf40b2cc0 con 0x7fbcf406d600 2024-03-08T06:05:15.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.767+0000 7fbcfaa00640 1 -- 172.21.15.40:0/3454568738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf40ad3d0 con 0x7fbcf406d600 2024-03-08T06:05:15.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.769+0000 7fbcebe00640 1 -- 172.21.15.40:0/3454568738 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbce401a070 con 0x7fbcf406d600 2024-03-08T06:05:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.770+0000 7fbcebe00640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbcd406b520 0x7fbcd406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.770+0000 7fbcebe00640 1 -- 172.21.15.40:0/3454568738 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbce4089620 con 0x7fbcf406d600 2024-03-08T06:05:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.770+0000 7fbcf3400640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbcd406b520 0x7fbcd406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.771+0000 7fbcf3400640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbcd406b520 0x7fbcd406d9e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbce0036000 tx=0x7fbce0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:15.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.775+0000 7fbcebe00640 1 -- 172.21.15.40:0/3454568738 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbce4058510 con 0x7fbcf406d600 2024-03-08T06:05:15.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.977+0000 7fbcfaa00640 1 -- 172.21.15.40:0/3454568738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcf4067cb0 con 0x7fbcf406d600 2024-03-08T06:05:15.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.979+0000 7fbcebe00640 1 -- 172.21.15.40:0/3454568738 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16585 (secure 0 0 0) 0x7fbce4057e10 con 0x7fbcf406d600 2024-03-08T06:05:15.979 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:05:15.980 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:05:12.978712+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:05:09.544904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T06:05:10.468274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:05:15.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.981+0000 7fbcfaa00640 1 -- 172.21.15.40:0/3454568738 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbcd406b520 msgr2=0x7fbcd406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.981+0000 7fbcfaa00640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbcd406b520 0x7fbcd406d9e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbce0036000 tx=0x7fbce0008030 comp rx=0 tx=0).stop 2024-03-08T06:05:15.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.981+0000 7fbcfaa00640 1 -- 172.21.15.40:0/3454568738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf406d600 msgr2=0x7fbcf40ace30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:15.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.981+0000 7fbcfaa00640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf406d600 0x7fbcf40ace30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbce400c400 tx=0x7fbce400c8c0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.982+0000 7fbcfaa00640 1 -- 172.21.15.40:0/3454568738 shutdown_connections 2024-03-08T06:05:15.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.982+0000 7fbcfaa00640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbcd406b520 0x7fbcd406d9e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.982+0000 7fbcfaa00640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbcf4071e90 0x7fbcf41f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.982+0000 7fbcfaa00640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcf4071ab0 0x7fbcf40cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.982+0000 7fbcfaa00640 1 --2- 172.21.15.40:0/3454568738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf406d600 0x7fbcf40ace30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:15.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.982+0000 7fbcfaa00640 1 -- 172.21.15.40:0/3454568738 >> 172.21.15.40:0/3454568738 conn(0x7fbcf406ae30 msgr2=0x7fbcf406b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:15.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.982+0000 7fbcfaa00640 1 -- 172.21.15.40:0/3454568738 shutdown_connections 2024-03-08T06:05:15.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:15.982+0000 7fbcfaa00640 1 -- 172.21.15.40:0/3454568738 wait complete. 2024-03-08T06:05:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-08T06:05:16.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:05:16.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.216+0000 7f3ab0c00640 1 Processor -- start 2024-03-08T06:05:16.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.216+0000 7f3ab0c00640 1 -- start start 2024-03-08T06:05:16.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.217+0000 7f3ab0c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3aac15e3b0 0x7f3aac166850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.217+0000 7f3ab0c00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac06da80 con 0x7f3aac071ab0 2024-03-08T06:05:16.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.217+0000 7f3ab0c00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac06dd60 con 0x7f3aac06d600 2024-03-08T06:05:16.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.217+0000 7f3ab0c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac06ef40 con 0x7f3aac15e3b0 2024-03-08T06:05:16.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.218+0000 7f3aa9e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3aac071ab0 0x7f3aac15cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43446/0 (socket says 172.21.15.40:43446) 2024-03-08T06:05:16.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.218+0000 7f3aa9e00640 1 -- 172.21.15.40:0/2030622493 learned_addr learned my addr 172.21.15.40:0/2030622493 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.218+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152187723 0 0) 0x7f3aac06da80 con 0x7f3aac071ab0 2024-03-08T06:05:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.218+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a90003520 con 0x7f3aac071ab0 2024-03-08T06:05:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.218+0000 7f3aaa800640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3aac15e3b0 0x7f3aac166850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1203327880 0 0) 0x7f3a90003520 con 0x7f3aac071ab0 2024-03-08T06:05:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3aac15e3b0 msgr2=0x7f3aac166850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3aac15e3b0 0x7f3aac166850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 >> v1:172.21.15.53:6789/0 conn(0x7f3aac06d600 legacy=0x7f3aac144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac06e040 con 0x7f3aac071ab0 2024-03-08T06:05:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f3a9c001670 con 0x7f3aac071ab0 2024-03-08T06:05:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 >> v1:172.21.15.40:6789/0 conn(0x7f3aac071ab0 legacy=0x7f3aac15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a900039e0 0x7f3a90003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3aac15e3b0 0x7f3a90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a90004d40 0x7f3a900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9000bb50 con 0x7f3a90004d40 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9000bcd0 con 0x7f3a900039e0 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9000bea0 con 0x7f3aac15e3b0 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aaa800640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a90004d40 0x7f3a900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.219+0000 7f3aa9400640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3aac15e3b0 0x7f3a90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.220+0000 7f3aa9e00640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a900039e0 0x7f3a90003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.220+0000 7f3aaa800640 1 -- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3aac15e3b0 msgr2=0x7f3a90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.220+0000 7f3aaa800640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3aac15e3b0 0x7f3a90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.220+0000 7f3aaa800640 1 -- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a900039e0 msgr2=0x7f3a90003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.220+0000 7f3aaa800640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a900039e0 0x7f3a90003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.220+0000 7f3aaa800640 1 -- 172.21.15.40:0/2030622493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a90003520 con 0x7f3a90004d40 2024-03-08T06:05:16.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.220+0000 7f3aa9e00640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a900039e0 0x7f3a90003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:16.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.221+0000 7f3aaa800640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a90004d40 0x7f3a900051c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4007360 tx=0x7f3a90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2546ca6e000d7755 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:16.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.221+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4039070 con 0x7f3a90004d40 2024-03-08T06:05:16.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.221+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa402cc80 con 0x7f3a90004d40 2024-03-08T06:05:16.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.221+0000 7f3aa8a00640 1 -- 172.21.15.40:0/2030622493 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4009be0 con 0x7f3a90004d40 2024-03-08T06:05:16.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.222+0000 7f3ab0c00640 1 -- 172.21.15.40:0/2030622493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a90004d40 msgr2=0x7f3a900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.222+0000 7f3ab0c00640 1 --2- 172.21.15.40:0/2030622493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a90004d40 0x7f3a900051c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4007360 tx=0x7f3a90005830 comp rx=0 tx=0).stop 2024-03-08T06:05:16.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.222+0000 7f3aaa800640 1 -- 172.21.15.40:0/2030622493 reap_dead start 2024-03-08T06:05:16.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.222+0000 7f3ab0c00640 1 -- 172.21.15.40:0/2030622493 shutdown_connections 2024-03-08T06:05:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.222+0000 7f3ab0c00640 1 -- 172.21.15.40:0/2030622493 >> 172.21.15.40:0/2030622493 conn(0x7f3aac06ae30 msgr2=0x7f3aac06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.222+0000 7f3ab0c00640 1 -- 172.21.15.40:0/2030622493 shutdown_connections 2024-03-08T06:05:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.222+0000 7f3ab0c00640 1 -- 172.21.15.40:0/2030622493 wait complete. 2024-03-08T06:05:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.224+0000 7f3ab0c00640 1 Processor -- start 2024-03-08T06:05:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.224+0000 7f3ab0c00640 1 -- start start 2024-03-08T06:05:16.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3ab0c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a90004d40 0x7f3aac0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3ab0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3aac130dd0 0x7f3aac1f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3ab0c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aac1323a0 0x7f3aac132820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3ab0c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac071e90 con 0x7f3aac1323a0 2024-03-08T06:05:16.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3ab0c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac06e600 con 0x7f3aac130dd0 2024-03-08T06:05:16.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3ab0c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac072010 con 0x7f3a90004d40 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3aaa800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aac1323a0 0x7f3aac132820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3aaa800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aac1323a0 0x7f3aac132820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33610/0 (socket says 172.21.15.40:33610) 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3aaa800640 1 -- 172.21.15.40:0/1344364519 learned_addr learned my addr 172.21.15.40:0/1344364519 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3aaa800640 1 -- 172.21.15.40:0/1344364519 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a90004d40 msgr2=0x7f3aac0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3aaa800640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a90004d40 0x7f3aac0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3aaa800640 1 -- 172.21.15.40:0/1344364519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3aac130dd0 msgr2=0x7f3aac1f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3aaa800640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3aac130dd0 0x7f3aac1f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.225+0000 7f3aaa800640 1 -- 172.21.15.40:0/1344364519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac131300 con 0x7f3aac1323a0 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.226+0000 7f3aaa800640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aac1323a0 0x7f3aac132820 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4007360 tx=0x7f3aa4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.226+0000 7f3aa2000640 1 -- 172.21.15.40:0/1344364519 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4039070 con 0x7f3aac1323a0 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.226+0000 7f3aa2000640 1 -- 172.21.15.40:0/1344364519 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa4044070 con 0x7f3aac1323a0 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.226+0000 7f3aa2000640 1 -- 172.21.15.40:0/1344364519 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4009be0 con 0x7f3aac1323a0 2024-03-08T06:05:16.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.226+0000 7f3ab0c00640 1 -- 172.21.15.40:0/1344364519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac1315e0 con 0x7f3aac1323a0 2024-03-08T06:05:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.226+0000 7f3ab0c00640 1 -- 172.21.15.40:0/1344364519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac131ba0 con 0x7f3aac1323a0 2024-03-08T06:05:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.227+0000 7f3ab0c00640 1 -- 172.21.15.40:0/1344364519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac067cb0 con 0x7f3aac1323a0 2024-03-08T06:05:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.228+0000 7f3aa2000640 1 -- 172.21.15.40:0/1344364519 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3aa403e830 con 0x7f3aac1323a0 2024-03-08T06:05:16.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.229+0000 7f3aa2000640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3a8c06b500 0x7f3a8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.229+0000 7f3aa2000640 1 -- 172.21.15.40:0/1344364519 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3aa40ac1c0 con 0x7f3aac1323a0 2024-03-08T06:05:16.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.229+0000 7f3aa9e00640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3a8c06b500 0x7f3a8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.230+0000 7f3aa9e00640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3a8c06b500 0x7f3a8c06d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c000f60 tx=0x7f3a9c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:16.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.234+0000 7f3aa2000640 1 -- 172.21.15.40:0/1344364519 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3aa407b040 con 0x7f3aac1323a0 2024-03-08T06:05:16.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.431+0000 7f3ab0c00640 1 -- 172.21.15.40:0/1344364519 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3aac06c5e0 con 0x7f3a8c06b500 2024-03-08T06:05:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.454+0000 7f3aa2000640 1 -- 172.21.15.40:0/1344364519 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251932 (secure 0 0 0) 0x7f3aac06c5e0 con 0x7f3a8c06b500 2024-03-08T06:05:16.459 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:05:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-03-08T06:05:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.460+0000 7f3ab0c00640 1 -- 172.21.15.40:0/1344364519 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3a8c06b500 msgr2=0x7f3a8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.460+0000 7f3ab0c00640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3a8c06b500 0x7f3a8c06d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c000f60 tx=0x7f3a9c009040 comp rx=0 tx=0).stop 2024-03-08T06:05:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.460+0000 7f3ab0c00640 1 -- 172.21.15.40:0/1344364519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aac1323a0 msgr2=0x7f3aac132820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.460+0000 7f3ab0c00640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aac1323a0 0x7f3aac132820 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4007360 tx=0x7f3aa4004ab0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.461+0000 7f3ab0c00640 1 -- 172.21.15.40:0/1344364519 shutdown_connections 2024-03-08T06:05:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.461+0000 7f3ab0c00640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3a8c06b500 0x7f3a8c06d9c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.461+0000 7f3ab0c00640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aac1323a0 0x7f3aac132820 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.461+0000 7f3ab0c00640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3aac130dd0 0x7f3aac1f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.461+0000 7f3ab0c00640 1 --2- 172.21.15.40:0/1344364519 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a90004d40 0x7f3aac0cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.461+0000 7f3ab0c00640 1 -- 172.21.15.40:0/1344364519 >> 172.21.15.40:0/1344364519 conn(0x7f3aac06ae30 msgr2=0x7f3aac15e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:16.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.461+0000 7f3ab0c00640 1 -- 172.21.15.40:0/1344364519 shutdown_connections 2024-03-08T06:05:16.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.461+0000 7f3ab0c00640 1 -- 172.21.15.40:0/1344364519 wait complete. 2024-03-08T06:05:16.482 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-03-08T06:05:15.506638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237032,"kb_used_data":4196,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737688,"statfs":{"total":773094113280,"available":772851392512,"internally_reserved":0,"allocated":4296704,"data_stored":2376952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.563319"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.998822+0000","last_change":"2024-03-08T06:05:12.000209+0000","last_active":"2024-03-08T06:05:12.998822+0000","last_peered":"2024-03-08T06:05:12.998822+0000","last_clean":"2024-03-08T06:05:12.998822+0000","last_became_active":"2024-03-08T06:05:11.999852+0000","last_became_peered":"2024-03-08T06:05:11.999852+0000","last_unstale":"2024-03-08T06:05:12.998822+0000","last_undegraded":"2024-03-08T06:05:12.998822+0000","last_fullsized":"2024-03-08T06:05:12.998822+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:29:12.650016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037831+0000","last_change":"2024-03-08T06:05:12.011066+0000","last_active":"2024-03-08T06:05:14.037831+0000","last_peered":"2024-03-08T06:05:14.037831+0000","last_clean":"2024-03-08T06:05:14.037831+0000","last_became_active":"2024-03-08T06:05:12.010672+0000","last_became_peered":"2024-03-08T06:05:12.010672+0000","last_unstale":"2024-03-08T06:05:14.037831+0000","last_undegraded":"2024-03-08T06:05:14.037831+0000","last_fullsized":"2024-03-08T06:05:14.037831+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:33:43.934376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.009061+0000","last_change":"2024-03-08T06:05:12.004749+0000","last_active":"2024-03-08T06:05:12.009061+0000","last_peered":"2024-03-08T06:05:12.009061+0000","last_clean":"2024-03-08T06:05:12.009061+0000","last_became_active":"2024-03-08T06:05:12.004384+0000","last_became_peered":"2024-03-08T06:05:12.004384+0000","last_unstale":"2024-03-08T06:05:12.009061+0000","last_undegraded":"2024-03-08T06:05:12.009061+0000","last_fullsized":"2024-03-08T06:05:12.009061+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:05:46.243688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895788+0000","last_change":"2024-03-08T06:05:12.001581+0000","last_active":"2024-03-08T06:05:13.895788+0000","last_peered":"2024-03-08T06:05:13.895788+0000","last_clean":"2024-03-08T06:05:13.895788+0000","last_became_active":"2024-03-08T06:05:12.000954+0000","last_became_peered":"2024-03-08T06:05:12.000954+0000","last_unstale":"2024-03-08T06:05:13.895788+0000","last_undegraded":"2024-03-08T06:05:13.895788+0000","last_fullsized":"2024-03-08T06:05:13.895788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:05:46.243688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.210604+0000","last_change":"2024-03-08T06:05:11.997690+0000","last_active":"2024-03-08T06:05:13.210604+0000","last_peered":"2024-03-08T06:05:13.210604+0000","last_clean":"2024-03-08T06:05:13.210604+0000","last_became_active":"2024-03-08T06:05:11.992959+0000","last_became_peered":"2024-03-08T06:05:11.992959+0000","last_unstale":"2024-03-08T06:05:13.210604+0000","last_undegraded":"2024-03-08T06:05:13.210604+0000","last_fullsized":"2024-03-08T06:05:13.210604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:46.016337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037318+0000","last_change":"2024-03-08T06:05:12.017054+0000","last_active":"2024-03-08T06:05:14.037318+0000","last_peered":"2024-03-08T06:05:14.037318+0000","last_clean":"2024-03-08T06:05:14.037318+0000","last_became_active":"2024-03-08T06:05:12.014998+0000","last_became_peered":"2024-03-08T06:05:12.014998+0000","last_unstale":"2024-03-08T06:05:14.037318+0000","last_undegraded":"2024-03-08T06:05:14.037318+0000","last_fullsized":"2024-03-08T06:05:14.037318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:01:00.803269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.997189+0000","last_change":"2024-03-08T06:05:12.002105+0000","last_active":"2024-03-08T06:05:12.997189+0000","last_peered":"2024-03-08T06:05:12.997189+0000","last_clean":"2024-03-08T06:05:12.997189+0000","last_became_active":"2024-03-08T06:05:12.001435+0000","last_became_peered":"2024-03-08T06:05:12.001435+0000","last_unstale":"2024-03-08T06:05:12.997189+0000","last_undegraded":"2024-03-08T06:05:12.997189+0000","last_fullsized":"2024-03-08T06:05:12.997189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:01:00.803269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996682+0000","last_change":"2024-03-08T06:05:12.008201+0000","last_active":"2024-03-08T06:05:12.996682+0000","last_peered":"2024-03-08T06:05:12.996682+0000","last_clean":"2024-03-08T06:05:12.996682+0000","last_became_active":"2024-03-08T06:05:12.007548+0000","last_became_peered":"2024-03-08T06:05:12.007548+0000","last_unstale":"2024-03-08T06:05:12.996682+0000","last_undegraded":"2024-03-08T06:05:12.996682+0000","last_fullsized":"2024-03-08T06:05:12.996682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:01.683645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.017542+0000","last_change":"2024-03-08T06:05:12.014267+0000","last_active":"2024-03-08T06:05:12.017542+0000","last_peered":"2024-03-08T06:05:12.017542+0000","last_clean":"2024-03-08T06:05:12.017542+0000","last_became_active":"2024-03-08T06:05:12.013448+0000","last_became_peered":"2024-03-08T06:05:12.013448+0000","last_unstale":"2024-03-08T06:05:12.017542+0000","last_undegraded":"2024-03-08T06:05:12.017542+0000","last_fullsized":"2024-03-08T06:05:12.017542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:08:43.342583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896446+0000","last_change":"2024-03-08T06:05:12.023314+0000","last_active":"2024-03-08T06:05:13.896446+0000","last_peered":"2024-03-08T06:05:13.896446+0000","last_clean":"2024-03-08T06:05:13.896446+0000","last_became_active":"2024-03-08T06:05:12.022826+0000","last_became_peered":"2024-03-08T06:05:12.022826+0000","last_unstale":"2024-03-08T06:05:13.896446+0000","last_undegraded":"2024-03-08T06:05:13.896446+0000","last_fullsized":"2024-03-08T06:05:13.896446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:11:41.014727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.997452+0000","last_change":"2024-03-08T06:05:12.011238+0000","last_active":"2024-03-08T06:05:12.997452+0000","last_peered":"2024-03-08T06:05:12.997452+0000","last_clean":"2024-03-08T06:05:12.997452+0000","last_became_active":"2024-03-08T06:05:12.010914+0000","last_became_peered":"2024-03-08T06:05:12.010914+0000","last_unstale":"2024-03-08T06:05:12.997452+0000","last_undegraded":"2024-03-08T06:05:12.997452+0000","last_fullsized":"2024-03-08T06:05:12.997452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:13:01.505273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.018481+0000","last_change":"2024-03-08T06:05:12.016727+0000","last_active":"2024-03-08T06:05:12.018481+0000","last_peered":"2024-03-08T06:05:12.018481+0000","last_clean":"2024-03-08T06:05:12.018481+0000","last_became_active":"2024-03-08T06:05:12.016419+0000","last_became_peered":"2024-03-08T06:05:12.016419+0000","last_unstale":"2024-03-08T06:05:12.018481+0000","last_undegraded":"2024-03-08T06:05:12.018481+0000","last_fullsized":"2024-03-08T06:05:12.018481+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:28:26.609404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.210615+0000","last_change":"2024-03-08T06:05:11.997914+0000","last_active":"2024-03-08T06:05:13.210615+0000","last_peered":"2024-03-08T06:05:13.210615+0000","last_clean":"2024-03-08T06:05:13.210615+0000","last_became_active":"2024-03-08T06:05:11.994055+0000","last_became_peered":"2024-03-08T06:05:11.994055+0000","last_unstale":"2024-03-08T06:05:13.210615+0000","last_undegraded":"2024-03-08T06:05:13.210615+0000","last_fullsized":"2024-03-08T06:05:13.210615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:42:12.730065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037726+0000","last_change":"2024-03-08T06:05:11.998899+0000","last_active":"2024-03-08T06:05:14.037726+0000","last_peered":"2024-03-08T06:05:14.037726+0000","last_clean":"2024-03-08T06:05:14.037726+0000","last_became_active":"2024-03-08T06:05:11.998604+0000","last_became_peered":"2024-03-08T06:05:11.998604+0000","last_unstale":"2024-03-08T06:05:14.037726+0000","last_undegraded":"2024-03-08T06:05:14.037726+0000","last_fullsized":"2024-03-08T06:05:14.037726+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:41:55.777936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.008189+0000","last_change":"2024-03-08T06:05:12.003117+0000","last_active":"2024-03-08T06:05:12.008189+0000","last_peered":"2024-03-08T06:05:12.008189+0000","last_clean":"2024-03-08T06:05:12.008189+0000","last_became_active":"2024-03-08T06:05:12.002805+0000","last_became_peered":"2024-03-08T06:05:12.002805+0000","last_unstale":"2024-03-08T06:05:12.008189+0000","last_undegraded":"2024-03-08T06:05:12.008189+0000","last_fullsized":"2024-03-08T06:05:12.008189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:16:58.567142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.997139+0000","last_change":"2024-03-08T06:05:12.008429+0000","last_active":"2024-03-08T06:05:12.997139+0000","last_peered":"2024-03-08T06:05:12.997139+0000","last_clean":"2024-03-08T06:05:12.997139+0000","last_became_active":"2024-03-08T06:05:12.007904+0000","last_became_peered":"2024-03-08T06:05:12.007904+0000","last_unstale":"2024-03-08T06:05:12.997139+0000","last_undegraded":"2024-03-08T06:05:12.997139+0000","last_fullsized":"2024-03-08T06:05:12.997139+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:38:39.993958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.997731+0000","last_change":"2024-03-08T06:05:12.006193+0000","last_active":"2024-03-08T06:05:12.997731+0000","last_peered":"2024-03-08T06:05:12.997731+0000","last_clean":"2024-03-08T06:05:12.997731+0000","last_became_active":"2024-03-08T06:05:12.004837+0000","last_became_peered":"2024-03-08T06:05:12.004837+0000","last_unstale":"2024-03-08T06:05:12.997731+0000","last_undegraded":"2024-03-08T06:05:12.997731+0000","last_fullsized":"2024-03-08T06:05:12.997731+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:12:21.419108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037635+0000","last_change":"2024-03-08T06:05:12.010653+0000","last_active":"2024-03-08T06:05:14.037635+0000","last_peered":"2024-03-08T06:05:14.037635+0000","last_clean":"2024-03-08T06:05:14.037635+0000","last_became_active":"2024-03-08T06:05:12.005042+0000","last_became_peered":"2024-03-08T06:05:12.005042+0000","last_unstale":"2024-03-08T06:05:14.037635+0000","last_undegraded":"2024-03-08T06:05:14.037635+0000","last_fullsized":"2024-03-08T06:05:14.037635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:12:21.419108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037548+0000","last_change":"2024-03-08T06:05:11.998689+0000","last_active":"2024-03-08T06:05:14.037548+0000","last_peered":"2024-03-08T06:05:14.037548+0000","last_clean":"2024-03-08T06:05:14.037548+0000","last_became_active":"2024-03-08T06:05:11.997155+0000","last_became_peered":"2024-03-08T06:05:11.997155+0000","last_unstale":"2024-03-08T06:05:14.037548+0000","last_undegraded":"2024-03-08T06:05:14.037548+0000","last_fullsized":"2024-03-08T06:05:14.037548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:13:01.505273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.211331+0000","last_change":"2024-03-08T06:05:11.989306+0000","last_active":"2024-03-08T06:05:13.211331+0000","last_peered":"2024-03-08T06:05:13.211331+0000","last_clean":"2024-03-08T06:05:13.211331+0000","last_became_active":"2024-03-08T06:05:11.989009+0000","last_became_peered":"2024-03-08T06:05:11.989009+0000","last_unstale":"2024-03-08T06:05:13.211331+0000","last_undegraded":"2024-03-08T06:05:13.211331+0000","last_fullsized":"2024-03-08T06:05:13.211331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:54:15.830517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.998836+0000","last_change":"2024-03-08T06:05:11.996110+0000","last_active":"2024-03-08T06:05:11.998836+0000","last_peered":"2024-03-08T06:05:11.998836+0000","last_clean":"2024-03-08T06:05:11.998836+0000","last_became_active":"2024-03-08T06:05:11.995775+0000","last_became_peered":"2024-03-08T06:05:11.995775+0000","last_unstale":"2024-03-08T06:05:11.998836+0000","last_undegraded":"2024-03-08T06:05:11.998836+0000","last_fullsized":"2024-03-08T06:05:11.998836+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:38:52.367265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.017100+0000","last_change":"2024-03-08T06:05:12.006289+0000","last_active":"2024-03-08T06:05:12.017100+0000","last_peered":"2024-03-08T06:05:12.017100+0000","last_clean":"2024-03-08T06:05:12.017100+0000","last_became_active":"2024-03-08T06:05:12.005925+0000","last_became_peered":"2024-03-08T06:05:12.005925+0000","last_unstale":"2024-03-08T06:05:12.017100+0000","last_undegraded":"2024-03-08T06:05:12.017100+0000","last_fullsized":"2024-03-08T06:05:12.017100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:46:20.340467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.710916+0000","last_change":"2024-03-08T06:05:12.007400+0000","last_active":"2024-03-08T06:05:13.710916+0000","last_peered":"2024-03-08T06:05:13.710916+0000","last_clean":"2024-03-08T06:05:13.710916+0000","last_became_active":"2024-03-08T06:05:12.007164+0000","last_became_peered":"2024-03-08T06:05:12.007164+0000","last_unstale":"2024-03-08T06:05:13.710916+0000","last_undegraded":"2024-03-08T06:05:13.710916+0000","last_fullsized":"2024-03-08T06:05:13.710916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:07:23.671282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.001545+0000","last_change":"2024-03-08T06:05:11.994250+0000","last_active":"2024-03-08T06:05:12.001545+0000","last_peered":"2024-03-08T06:05:12.001545+0000","last_clean":"2024-03-08T06:05:12.001545+0000","last_became_active":"2024-03-08T06:05:11.993847+0000","last_became_peered":"2024-03-08T06:05:11.993847+0000","last_unstale":"2024-03-08T06:05:12.001545+0000","last_undegraded":"2024-03-08T06:05:12.001545+0000","last_fullsized":"2024-03-08T06:05:12.001545+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:07:23.671282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896825+0000","last_change":"2024-03-08T06:05:12.009917+0000","last_active":"2024-03-08T06:05:13.896825+0000","last_peered":"2024-03-08T06:05:13.896825+0000","last_clean":"2024-03-08T06:05:13.896825+0000","last_became_active":"2024-03-08T06:05:12.009672+0000","last_became_peered":"2024-03-08T06:05:12.009672+0000","last_unstale":"2024-03-08T06:05:13.896825+0000","last_undegraded":"2024-03-08T06:05:13.896825+0000","last_fullsized":"2024-03-08T06:05:13.896825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:28:26.609404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037815+0000","last_change":"2024-03-08T06:05:12.010910+0000","last_active":"2024-03-08T06:05:14.037815+0000","last_peered":"2024-03-08T06:05:14.037815+0000","last_clean":"2024-03-08T06:05:14.037815+0000","last_became_active":"2024-03-08T06:05:12.006915+0000","last_became_peered":"2024-03-08T06:05:12.006915+0000","last_unstale":"2024-03-08T06:05:14.037815+0000","last_undegraded":"2024-03-08T06:05:14.037815+0000","last_fullsized":"2024-03-08T06:05:14.037815+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:14:01.815309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896408+0000","last_change":"2024-03-08T06:05:11.999823+0000","last_active":"2024-03-08T06:05:13.896408+0000","last_peered":"2024-03-08T06:05:13.896408+0000","last_clean":"2024-03-08T06:05:13.896408+0000","last_became_active":"2024-03-08T06:05:11.999131+0000","last_became_peered":"2024-03-08T06:05:11.999131+0000","last_unstale":"2024-03-08T06:05:13.896408+0000","last_undegraded":"2024-03-08T06:05:13.896408+0000","last_fullsized":"2024-03-08T06:05:13.896408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:19:47.069509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.001933+0000","last_change":"2024-03-08T06:05:11.996383+0000","last_active":"2024-03-08T06:05:12.001933+0000","last_peered":"2024-03-08T06:05:12.001933+0000","last_clean":"2024-03-08T06:05:12.001933+0000","last_became_active":"2024-03-08T06:05:11.996081+0000","last_became_peered":"2024-03-08T06:05:11.996081+0000","last_unstale":"2024-03-08T06:05:12.001933+0000","last_undegraded":"2024-03-08T06:05:12.001933+0000","last_fullsized":"2024-03-08T06:05:12.001933+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:01:49.887776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.015733+0000","last_change":"2024-03-08T06:05:12.005994+0000","last_active":"2024-03-08T06:05:12.015733+0000","last_peered":"2024-03-08T06:05:12.015733+0000","last_clean":"2024-03-08T06:05:12.015733+0000","last_became_active":"2024-03-08T06:05:12.005673+0000","last_became_peered":"2024-03-08T06:05:12.005673+0000","last_unstale":"2024-03-08T06:05:12.015733+0000","last_undegraded":"2024-03-08T06:05:12.015733+0000","last_fullsized":"2024-03-08T06:05:12.015733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:48:49.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996584+0000","last_change":"2024-03-08T06:05:12.001065+0000","last_active":"2024-03-08T06:05:12.996584+0000","last_peered":"2024-03-08T06:05:12.996584+0000","last_clean":"2024-03-08T06:05:12.996584+0000","last_became_active":"2024-03-08T06:05:12.000571+0000","last_became_peered":"2024-03-08T06:05:12.000571+0000","last_unstale":"2024-03-08T06:05:12.996584+0000","last_undegraded":"2024-03-08T06:05:12.996584+0000","last_fullsized":"2024-03-08T06:05:12.996584+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:17:38.904380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.998792+0000","last_change":"2024-03-08T06:05:11.990653+0000","last_active":"2024-03-08T06:05:12.998792+0000","last_peered":"2024-03-08T06:05:12.998792+0000","last_clean":"2024-03-08T06:05:12.998792+0000","last_became_active":"2024-03-08T06:05:11.990061+0000","last_became_peered":"2024-03-08T06:05:11.990061+0000","last_unstale":"2024-03-08T06:05:12.998792+0000","last_undegraded":"2024-03-08T06:05:12.998792+0000","last_fullsized":"2024-03-08T06:05:12.998792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:38:38.055804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.995966+0000","last_change":"2024-03-08T06:05:12.007166+0000","last_active":"2024-03-08T06:05:12.995966+0000","last_peered":"2024-03-08T06:05:12.995966+0000","last_clean":"2024-03-08T06:05:12.995966+0000","last_became_active":"2024-03-08T06:05:12.003713+0000","last_became_peered":"2024-03-08T06:05:12.003713+0000","last_unstale":"2024-03-08T06:05:12.995966+0000","last_undegraded":"2024-03-08T06:05:12.995966+0000","last_fullsized":"2024-03-08T06:05:12.995966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:21:16.418811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.711201+0000","last_change":"2024-03-08T06:05:12.007927+0000","last_active":"2024-03-08T06:05:13.711201+0000","last_peered":"2024-03-08T06:05:13.711201+0000","last_clean":"2024-03-08T06:05:13.711201+0000","last_became_active":"2024-03-08T06:05:12.007457+0000","last_became_peered":"2024-03-08T06:05:12.007457+0000","last_unstale":"2024-03-08T06:05:13.711201+0000","last_undegraded":"2024-03-08T06:05:13.711201+0000","last_fullsized":"2024-03-08T06:05:13.711201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:45:29.983658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.999163+0000","last_change":"2024-03-08T06:05:10.999998+0000","last_active":"2024-03-08T06:05:12.999163+0000","last_peered":"2024-03-08T06:05:12.999163+0000","last_clean":"2024-03-08T06:05:12.999163+0000","last_became_active":"2024-03-08T06:05:10.999712+0000","last_became_peered":"2024-03-08T06:05:10.999712+0000","last_unstale":"2024-03-08T06:05:12.999163+0000","last_undegraded":"2024-03-08T06:05:12.999163+0000","last_fullsized":"2024-03-08T06:05:12.999163+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:54:14.815255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.999159+0000","last_change":"2024-03-08T06:05:11.998105+0000","last_active":"2024-03-08T06:05:12.999159+0000","last_peered":"2024-03-08T06:05:12.999159+0000","last_clean":"2024-03-08T06:05:12.999159+0000","last_became_active":"2024-03-08T06:05:11.997785+0000","last_became_peered":"2024-03-08T06:05:11.997785+0000","last_unstale":"2024-03-08T06:05:12.999159+0000","last_undegraded":"2024-03-08T06:05:12.999159+0000","last_fullsized":"2024-03-08T06:05:12.999159+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:46:30.021879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.711315+0000","last_change":"2024-03-08T06:05:10.996010+0000","last_active":"2024-03-08T06:05:13.711315+0000","last_peered":"2024-03-08T06:05:13.711315+0000","last_clean":"2024-03-08T06:05:13.711315+0000","last_became_active":"2024-03-08T06:05:10.995723+0000","last_became_peered":"2024-03-08T06:05:10.995723+0000","last_unstale":"2024-03-08T06:05:13.711315+0000","last_undegraded":"2024-03-08T06:05:13.711315+0000","last_fullsized":"2024-03-08T06:05:13.711315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:48:48.169538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.015669+0000","last_change":"2024-03-08T06:05:12.005945+0000","last_active":"2024-03-08T06:05:12.015669+0000","last_peered":"2024-03-08T06:05:12.015669+0000","last_clean":"2024-03-08T06:05:12.015669+0000","last_became_active":"2024-03-08T06:05:12.005589+0000","last_became_peered":"2024-03-08T06:05:12.005589+0000","last_unstale":"2024-03-08T06:05:12.015669+0000","last_undegraded":"2024-03-08T06:05:12.015669+0000","last_fullsized":"2024-03-08T06:05:12.015669+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:06:51.338297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037331+0000","last_change":"2024-03-08T06:05:10.990927+0000","last_active":"2024-03-08T06:05:14.037331+0000","last_peered":"2024-03-08T06:05:14.037331+0000","last_clean":"2024-03-08T06:05:14.037331+0000","last_became_active":"2024-03-08T06:05:10.990580+0000","last_became_peered":"2024-03-08T06:05:10.990580+0000","last_unstale":"2024-03-08T06:05:14.037331+0000","last_undegraded":"2024-03-08T06:05:14.037331+0000","last_fullsized":"2024-03-08T06:05:14.037331+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:46:29.006617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896751+0000","last_change":"2024-03-08T06:05:12.002689+0000","last_active":"2024-03-08T06:05:13.896751+0000","last_peered":"2024-03-08T06:05:13.896751+0000","last_clean":"2024-03-08T06:05:13.896751+0000","last_became_active":"2024-03-08T06:05:11.999478+0000","last_became_peered":"2024-03-08T06:05:11.999478+0000","last_unstale":"2024-03-08T06:05:13.896751+0000","last_undegraded":"2024-03-08T06:05:13.896751+0000","last_fullsized":"2024-03-08T06:05:13.896751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:21:16.418811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.211272+0000","last_change":"2024-03-08T06:05:10.993573+0000","last_active":"2024-03-08T06:05:13.211272+0000","last_peered":"2024-03-08T06:05:13.211272+0000","last_clean":"2024-03-08T06:05:13.211272+0000","last_became_active":"2024-03-08T06:05:10.993235+0000","last_became_peered":"2024-03-08T06:05:10.993235+0000","last_unstale":"2024-03-08T06:05:13.211272+0000","last_undegraded":"2024-03-08T06:05:13.211272+0000","last_fullsized":"2024-03-08T06:05:13.211272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:42:51.826683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996702+0000","last_change":"2024-03-08T06:05:12.000803+0000","last_active":"2024-03-08T06:05:12.996702+0000","last_peered":"2024-03-08T06:05:12.996702+0000","last_clean":"2024-03-08T06:05:12.996702+0000","last_became_active":"2024-03-08T06:05:12.000044+0000","last_became_peered":"2024-03-08T06:05:12.000044+0000","last_unstale":"2024-03-08T06:05:12.996702+0000","last_undegraded":"2024-03-08T06:05:12.996702+0000","last_fullsized":"2024-03-08T06:05:12.996702+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:33:12.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.022729+0000","last_change":"2024-03-08T06:05:10.986207+0000","last_active":"2024-03-08T06:05:12.022729+0000","last_peered":"2024-03-08T06:05:12.022729+0000","last_clean":"2024-03-08T06:05:12.022729+0000","last_became_active":"2024-03-08T06:05:10.985748+0000","last_became_peered":"2024-03-08T06:05:10.985748+0000","last_unstale":"2024-03-08T06:05:12.022729+0000","last_undegraded":"2024-03-08T06:05:12.022729+0000","last_fullsized":"2024-03-08T06:05:12.022729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:13:49.466728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.011933+0000","last_change":"2024-03-08T06:05:12.007888+0000","last_active":"2024-03-08T06:05:12.011933+0000","last_peered":"2024-03-08T06:05:12.011933+0000","last_clean":"2024-03-08T06:05:12.011933+0000","last_became_active":"2024-03-08T06:05:12.007570+0000","last_became_peered":"2024-03-08T06:05:12.007570+0000","last_unstale":"2024-03-08T06:05:12.011933+0000","last_undegraded":"2024-03-08T06:05:12.011933+0000","last_fullsized":"2024-03-08T06:05:12.011933+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:19:47.069509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.022778+0000","last_change":"2024-03-08T06:05:10.978505+0000","last_active":"2024-03-08T06:05:12.022778+0000","last_peered":"2024-03-08T06:05:12.022778+0000","last_clean":"2024-03-08T06:05:12.022778+0000","last_became_active":"2024-03-08T06:05:10.978175+0000","last_became_peered":"2024-03-08T06:05:10.978175+0000","last_unstale":"2024-03-08T06:05:12.022778+0000","last_undegraded":"2024-03-08T06:05:12.022778+0000","last_fullsized":"2024-03-08T06:05:12.022778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:01:32.412422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.003404+0000","last_change":"2024-03-08T06:05:11.998586+0000","last_active":"2024-03-08T06:05:12.003404+0000","last_peered":"2024-03-08T06:05:12.003404+0000","last_clean":"2024-03-08T06:05:12.003404+0000","last_became_active":"2024-03-08T06:05:11.998259+0000","last_became_peered":"2024-03-08T06:05:11.998259+0000","last_unstale":"2024-03-08T06:05:12.003404+0000","last_undegraded":"2024-03-08T06:05:12.003404+0000","last_fullsized":"2024-03-08T06:05:12.003404+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:54:15.830517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036635+0000","last_change":"2024-03-08T06:05:10.982867+0000","last_active":"2024-03-08T06:05:14.036635+0000","last_peered":"2024-03-08T06:05:14.036635+0000","last_clean":"2024-03-08T06:05:14.036635+0000","last_became_active":"2024-03-08T06:05:10.979795+0000","last_became_peered":"2024-03-08T06:05:10.979795+0000","last_unstale":"2024-03-08T06:05:14.036635+0000","last_undegraded":"2024-03-08T06:05:14.036635+0000","last_fullsized":"2024-03-08T06:05:14.036635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:17:36.557421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895303+0000","last_change":"2024-03-08T06:05:12.002205+0000","last_active":"2024-03-08T06:05:13.895303+0000","last_peered":"2024-03-08T06:05:13.895303+0000","last_clean":"2024-03-08T06:05:13.895303+0000","last_became_active":"2024-03-08T06:05:12.001597+0000","last_became_peered":"2024-03-08T06:05:12.001597+0000","last_unstale":"2024-03-08T06:05:13.895303+0000","last_undegraded":"2024-03-08T06:05:13.895303+0000","last_fullsized":"2024-03-08T06:05:13.895303+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:45:29.983658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.982497+0000","last_change":"2024-03-08T06:05:10.982065+0000","last_active":"2024-03-08T06:05:11.982497+0000","last_peered":"2024-03-08T06:05:11.982497+0000","last_clean":"2024-03-08T06:05:11.982497+0000","last_became_active":"2024-03-08T06:05:10.981682+0000","last_became_peered":"2024-03-08T06:05:10.981682+0000","last_unstale":"2024-03-08T06:05:11.982497+0000","last_undegraded":"2024-03-08T06:05:11.982497+0000","last_fullsized":"2024-03-08T06:05:11.982497+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:45.001075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.018446+0000","last_change":"2024-03-08T06:05:12.016443+0000","last_active":"2024-03-08T06:05:12.018446+0000","last_peered":"2024-03-08T06:05:12.018446+0000","last_clean":"2024-03-08T06:05:12.018446+0000","last_became_active":"2024-03-08T06:05:12.016114+0000","last_became_peered":"2024-03-08T06:05:12.016114+0000","last_unstale":"2024-03-08T06:05:12.018446+0000","last_undegraded":"2024-03-08T06:05:12.018446+0000","last_fullsized":"2024-03-08T06:05:12.018446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:21:16.418811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037792+0000","last_change":"2024-03-08T06:05:10.994763+0000","last_active":"2024-03-08T06:05:14.037792+0000","last_peered":"2024-03-08T06:05:14.037792+0000","last_clean":"2024-03-08T06:05:14.037792+0000","last_became_active":"2024-03-08T06:05:10.994452+0000","last_became_peered":"2024-03-08T06:05:10.994452+0000","last_unstale":"2024-03-08T06:05:14.037792+0000","last_undegraded":"2024-03-08T06:05:14.037792+0000","last_fullsized":"2024-03-08T06:05:14.037792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:01:48.872514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037675+0000","last_change":"2024-03-08T06:05:11.999617+0000","last_active":"2024-03-08T06:05:14.037675+0000","last_peered":"2024-03-08T06:05:14.037675+0000","last_clean":"2024-03-08T06:05:14.037675+0000","last_became_active":"2024-03-08T06:05:11.998297+0000","last_became_peered":"2024-03-08T06:05:11.998297+0000","last_unstale":"2024-03-08T06:05:14.037675+0000","last_undegraded":"2024-03-08T06:05:14.037675+0000","last_fullsized":"2024-03-08T06:05:14.037675+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:50:05.691691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.711497+0000","last_change":"2024-03-08T06:05:10.986943+0000","last_active":"2024-03-08T06:05:13.711497+0000","last_peered":"2024-03-08T06:05:13.711497+0000","last_clean":"2024-03-08T06:05:13.711497+0000","last_became_active":"2024-03-08T06:05:10.985728+0000","last_became_peered":"2024-03-08T06:05:10.985728+0000","last_unstale":"2024-03-08T06:05:13.711497+0000","last_undegraded":"2024-03-08T06:05:13.711497+0000","last_fullsized":"2024-03-08T06:05:13.711497+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:50:59.391540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037204+0000","last_change":"2024-03-08T06:05:12.007778+0000","last_active":"2024-03-08T06:05:14.037204+0000","last_peered":"2024-03-08T06:05:14.037204+0000","last_clean":"2024-03-08T06:05:14.037204+0000","last_became_active":"2024-03-08T06:05:12.007420+0000","last_became_peered":"2024-03-08T06:05:12.007420+0000","last_unstale":"2024-03-08T06:05:14.037204+0000","last_undegraded":"2024-03-08T06:05:14.037204+0000","last_fullsized":"2024-03-08T06:05:14.037204+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:38:39.993958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.018619+0000","last_change":"2024-03-08T06:05:10.999021+0000","last_active":"2024-03-08T06:05:12.018619+0000","last_peered":"2024-03-08T06:05:12.018619+0000","last_clean":"2024-03-08T06:05:12.018619+0000","last_became_active":"2024-03-08T06:05:10.998545+0000","last_became_peered":"2024-03-08T06:05:10.998545+0000","last_unstale":"2024-03-08T06:05:12.018619+0000","last_undegraded":"2024-03-08T06:05:12.018619+0000","last_fullsized":"2024-03-08T06:05:12.018619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:47.068546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.015422+0000","last_change":"2024-03-08T06:05:12.011129+0000","last_active":"2024-03-08T06:05:12.015422+0000","last_peered":"2024-03-08T06:05:12.015422+0000","last_clean":"2024-03-08T06:05:12.015422+0000","last_became_active":"2024-03-08T06:05:12.010810+0000","last_became_peered":"2024-03-08T06:05:12.010810+0000","last_unstale":"2024-03-08T06:05:12.015422+0000","last_undegraded":"2024-03-08T06:05:12.015422+0000","last_fullsized":"2024-03-08T06:05:12.015422+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:11:41.014727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037366+0000","last_change":"2024-03-08T06:05:10.991380+0000","last_active":"2024-03-08T06:05:14.037366+0000","last_peered":"2024-03-08T06:05:14.037366+0000","last_clean":"2024-03-08T06:05:14.037366+0000","last_became_active":"2024-03-08T06:05:10.991091+0000","last_became_peered":"2024-03-08T06:05:10.991091+0000","last_unstale":"2024-03-08T06:05:14.037366+0000","last_undegraded":"2024-03-08T06:05:14.037366+0000","last_fullsized":"2024-03-08T06:05:14.037366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:29:11.634754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895422+0000","last_change":"2024-03-08T06:05:12.000194+0000","last_active":"2024-03-08T06:05:13.895422+0000","last_peered":"2024-03-08T06:05:13.895422+0000","last_clean":"2024-03-08T06:05:13.895422+0000","last_became_active":"2024-03-08T06:05:11.999934+0000","last_became_peered":"2024-03-08T06:05:11.999934+0000","last_unstale":"2024-03-08T06:05:13.895422+0000","last_undegraded":"2024-03-08T06:05:13.895422+0000","last_fullsized":"2024-03-08T06:05:13.895422+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:16:58.567142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.017578+0000","last_change":"2024-03-08T06:05:10.984763+0000","last_active":"2024-03-08T06:05:12.017578+0000","last_peered":"2024-03-08T06:05:12.017578+0000","last_clean":"2024-03-08T06:05:12.017578+0000","last_became_active":"2024-03-08T06:05:10.983755+0000","last_became_peered":"2024-03-08T06:05:10.983755+0000","last_unstale":"2024-03-08T06:05:12.017578+0000","last_undegraded":"2024-03-08T06:05:12.017578+0000","last_fullsized":"2024-03-08T06:05:12.017578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:40:13.164263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036960+0000","last_change":"2024-03-08T06:05:11.994532+0000","last_active":"2024-03-08T06:05:14.036960+0000","last_peered":"2024-03-08T06:05:14.036960+0000","last_clean":"2024-03-08T06:05:14.036960+0000","last_became_active":"2024-03-08T06:05:11.994237+0000","last_became_peered":"2024-03-08T06:05:11.994237+0000","last_unstale":"2024-03-08T06:05:14.036960+0000","last_undegraded":"2024-03-08T06:05:14.036960+0000","last_fullsized":"2024-03-08T06:05:14.036960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:33:12.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895340+0000","last_change":"2024-03-08T06:05:10.989606+0000","last_active":"2024-03-08T06:05:13.895340+0000","last_peered":"2024-03-08T06:05:13.895340+0000","last_clean":"2024-03-08T06:05:13.895340+0000","last_became_active":"2024-03-08T06:05:10.989316+0000","last_became_peered":"2024-03-08T06:05:10.989316+0000","last_unstale":"2024-03-08T06:05:13.895340+0000","last_undegraded":"2024-03-08T06:05:13.895340+0000","last_fullsized":"2024-03-08T06:05:13.895340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:54:14.815255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996585+0000","last_change":"2024-03-08T06:05:12.003091+0000","last_active":"2024-03-08T06:05:12.996585+0000","last_peered":"2024-03-08T06:05:12.996585+0000","last_clean":"2024-03-08T06:05:12.996585+0000","last_became_active":"2024-03-08T06:05:12.002717+0000","last_became_peered":"2024-03-08T06:05:12.002717+0000","last_unstale":"2024-03-08T06:05:12.996585+0000","last_undegraded":"2024-03-08T06:05:12.996585+0000","last_fullsized":"2024-03-08T06:05:12.996585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:46:44.252156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036582+0000","last_change":"2024-03-08T06:05:10.994819+0000","last_active":"2024-03-08T06:05:14.036582+0000","last_peered":"2024-03-08T06:05:14.036582+0000","last_clean":"2024-03-08T06:05:14.036582+0000","last_became_active":"2024-03-08T06:05:10.994555+0000","last_became_peered":"2024-03-08T06:05:10.994555+0000","last_unstale":"2024-03-08T06:05:14.036582+0000","last_undegraded":"2024-03-08T06:05:14.036582+0000","last_fullsized":"2024-03-08T06:05:14.036582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:38:37.040542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.999155+0000","last_change":"2024-03-08T06:05:12.001069+0000","last_active":"2024-03-08T06:05:12.999155+0000","last_peered":"2024-03-08T06:05:12.999155+0000","last_clean":"2024-03-08T06:05:12.999155+0000","last_became_active":"2024-03-08T06:05:12.000761+0000","last_became_peered":"2024-03-08T06:05:12.000761+0000","last_unstale":"2024-03-08T06:05:12.999155+0000","last_undegraded":"2024-03-08T06:05:12.999155+0000","last_fullsized":"2024-03-08T06:05:12.999155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:48.083808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036586+0000","last_change":"2024-03-08T06:05:10.990848+0000","last_active":"2024-03-08T06:05:14.036586+0000","last_peered":"2024-03-08T06:05:14.036586+0000","last_clean":"2024-03-08T06:05:14.036586+0000","last_became_active":"2024-03-08T06:05:10.990537+0000","last_became_peered":"2024-03-08T06:05:10.990537+0000","last_unstale":"2024-03-08T06:05:14.036586+0000","last_undegraded":"2024-03-08T06:05:14.036586+0000","last_fullsized":"2024-03-08T06:05:14.036586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:47.068546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.015559+0000","last_change":"2024-03-08T06:05:12.004997+0000","last_active":"2024-03-08T06:05:12.015559+0000","last_peered":"2024-03-08T06:05:12.015559+0000","last_clean":"2024-03-08T06:05:12.015559+0000","last_became_active":"2024-03-08T06:05:12.004542+0000","last_became_peered":"2024-03-08T06:05:12.004542+0000","last_unstale":"2024-03-08T06:05:12.015559+0000","last_undegraded":"2024-03-08T06:05:12.015559+0000","last_fullsized":"2024-03-08T06:05:12.015559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:34:05.214759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896774+0000","last_change":"2024-03-08T06:05:10.986357+0000","last_active":"2024-03-08T06:05:13.896774+0000","last_peered":"2024-03-08T06:05:13.896774+0000","last_clean":"2024-03-08T06:05:13.896774+0000","last_became_active":"2024-03-08T06:05:10.986053+0000","last_became_peered":"2024-03-08T06:05:10.986053+0000","last_unstale":"2024-03-08T06:05:13.896774+0000","last_undegraded":"2024-03-08T06:05:13.896774+0000","last_fullsized":"2024-03-08T06:05:13.896774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:06:50.323035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.002475+0000","last_change":"2024-03-08T06:05:11.999467+0000","last_active":"2024-03-08T06:05:12.002475+0000","last_peered":"2024-03-08T06:05:12.002475+0000","last_clean":"2024-03-08T06:05:12.002475+0000","last_became_active":"2024-03-08T06:05:11.999191+0000","last_became_peered":"2024-03-08T06:05:11.999191+0000","last_unstale":"2024-03-08T06:05:12.002475+0000","last_undegraded":"2024-03-08T06:05:12.002475+0000","last_fullsized":"2024-03-08T06:05:12.002475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:45:29.983658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.017582+0000","last_change":"2024-03-08T06:05:10.987573+0000","last_active":"2024-03-08T06:05:12.017582+0000","last_peered":"2024-03-08T06:05:12.017582+0000","last_clean":"2024-03-08T06:05:12.017582+0000","last_became_active":"2024-03-08T06:05:10.985656+0000","last_became_peered":"2024-03-08T06:05:10.985656+0000","last_unstale":"2024-03-08T06:05:12.017582+0000","last_undegraded":"2024-03-08T06:05:12.017582+0000","last_fullsized":"2024-03-08T06:05:12.017582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:44:58.998250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895886+0000","last_change":"2024-03-08T06:05:12.001244+0000","last_active":"2024-03-08T06:05:13.895886+0000","last_peered":"2024-03-08T06:05:13.895886+0000","last_clean":"2024-03-08T06:05:13.895886+0000","last_became_active":"2024-03-08T06:05:12.000937+0000","last_became_peered":"2024-03-08T06:05:12.000937+0000","last_unstale":"2024-03-08T06:05:13.895886+0000","last_undegraded":"2024-03-08T06:05:13.895886+0000","last_fullsized":"2024-03-08T06:05:13.895886+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:57:44.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.017410+0000","last_change":"2024-03-08T06:05:10.980519+0000","last_active":"2024-03-08T06:05:12.017410+0000","last_peered":"2024-03-08T06:05:12.017410+0000","last_clean":"2024-03-08T06:05:12.017410+0000","last_became_active":"2024-03-08T06:05:10.980234+0000","last_became_peered":"2024-03-08T06:05:10.980234+0000","last_unstale":"2024-03-08T06:05:12.017410+0000","last_undegraded":"2024-03-08T06:05:12.017410+0000","last_fullsized":"2024-03-08T06:05:12.017410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:31:48.237166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.211054+0000","last_change":"2024-03-08T06:05:12.003229+0000","last_active":"2024-03-08T06:05:13.211054+0000","last_peered":"2024-03-08T06:05:13.211054+0000","last_clean":"2024-03-08T06:05:13.211054+0000","last_became_active":"2024-03-08T06:05:12.002830+0000","last_became_peered":"2024-03-08T06:05:12.002830+0000","last_unstale":"2024-03-08T06:05:13.211054+0000","last_undegraded":"2024-03-08T06:05:13.211054+0000","last_fullsized":"2024-03-08T06:05:13.211054+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:42:25.705238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.991981+0000","last_change":"2024-03-08T06:05:10.988057+0000","last_active":"2024-03-08T06:05:11.991981+0000","last_peered":"2024-03-08T06:05:11.991981+0000","last_clean":"2024-03-08T06:05:11.991981+0000","last_became_active":"2024-03-08T06:05:10.984660+0000","last_became_peered":"2024-03-08T06:05:10.984660+0000","last_unstale":"2024-03-08T06:05:11.991981+0000","last_undegraded":"2024-03-08T06:05:11.991981+0000","last_fullsized":"2024-03-08T06:05:11.991981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:42:24.689976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.210677+0000","last_change":"2024-03-08T06:05:11.991397+0000","last_active":"2024-03-08T06:05:13.210677+0000","last_peered":"2024-03-08T06:05:13.210677+0000","last_clean":"2024-03-08T06:05:13.210677+0000","last_became_active":"2024-03-08T06:05:11.991120+0000","last_became_peered":"2024-03-08T06:05:11.991120+0000","last_unstale":"2024-03-08T06:05:13.210677+0000","last_undegraded":"2024-03-08T06:05:13.210677+0000","last_fullsized":"2024-03-08T06:05:13.210677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:44.471571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.710776+0000","last_change":"2024-03-08T06:05:10.985889+0000","last_active":"2024-03-08T06:05:13.710776+0000","last_peered":"2024-03-08T06:05:13.710776+0000","last_clean":"2024-03-08T06:05:13.710776+0000","last_became_active":"2024-03-08T06:05:10.985583+0000","last_became_peered":"2024-03-08T06:05:10.985583+0000","last_unstale":"2024-03-08T06:05:13.710776+0000","last_undegraded":"2024-03-08T06:05:13.710776+0000","last_fullsized":"2024-03-08T06:05:13.710776+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:34:04.199497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.211158+0000","last_change":"2024-03-08T06:05:11.997394+0000","last_active":"2024-03-08T06:05:13.211158+0000","last_peered":"2024-03-08T06:05:13.211158+0000","last_clean":"2024-03-08T06:05:13.211158+0000","last_became_active":"2024-03-08T06:05:11.993710+0000","last_became_peered":"2024-03-08T06:05:11.993710+0000","last_unstale":"2024-03-08T06:05:13.211158+0000","last_undegraded":"2024-03-08T06:05:13.211158+0000","last_fullsized":"2024-03-08T06:05:13.211158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:38:52.367265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.998439+0000","last_change":"2024-03-08T06:05:10.985463+0000","last_active":"2024-03-08T06:05:12.998439+0000","last_peered":"2024-03-08T06:05:12.998439+0000","last_clean":"2024-03-08T06:05:12.998439+0000","last_became_active":"2024-03-08T06:05:10.985175+0000","last_became_peered":"2024-03-08T06:05:10.985175+0000","last_unstale":"2024-03-08T06:05:12.998439+0000","last_undegraded":"2024-03-08T06:05:12.998439+0000","last_fullsized":"2024-03-08T06:05:12.998439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:38:51.352003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.210751+0000","last_change":"2024-03-08T06:05:11.997488+0000","last_active":"2024-03-08T06:05:13.210751+0000","last_peered":"2024-03-08T06:05:13.210751+0000","last_clean":"2024-03-08T06:05:13.210751+0000","last_became_active":"2024-03-08T06:05:11.995546+0000","last_became_peered":"2024-03-08T06:05:11.995546+0000","last_unstale":"2024-03-08T06:05:13.210751+0000","last_undegraded":"2024-03-08T06:05:13.210751+0000","last_fullsized":"2024-03-08T06:05:13.210751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:31:49.252429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.986652+0000","last_change":"2024-03-08T06:05:10.987956+0000","last_active":"2024-03-08T06:05:11.986652+0000","last_peered":"2024-03-08T06:05:11.986652+0000","last_clean":"2024-03-08T06:05:11.986652+0000","last_became_active":"2024-03-08T06:05:10.987520+0000","last_became_peered":"2024-03-08T06:05:10.987520+0000","last_unstale":"2024-03-08T06:05:11.986652+0000","last_undegraded":"2024-03-08T06:05:11.986652+0000","last_fullsized":"2024-03-08T06:05:11.986652+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:42:11.714803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996706+0000","last_change":"2024-03-08T06:05:12.005301+0000","last_active":"2024-03-08T06:05:12.996706+0000","last_peered":"2024-03-08T06:05:12.996706+0000","last_clean":"2024-03-08T06:05:12.996706+0000","last_became_active":"2024-03-08T06:05:12.004898+0000","last_became_peered":"2024-03-08T06:05:12.004898+0000","last_unstale":"2024-03-08T06:05:12.996706+0000","last_undegraded":"2024-03-08T06:05:12.996706+0000","last_fullsized":"2024-03-08T06:05:12.996706+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:23:46.040146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.978417+0000","last_change":"2024-03-08T06:05:10.998091+0000","last_active":"2024-03-08T06:05:11.978417+0000","last_peered":"2024-03-08T06:05:11.978417+0000","last_clean":"2024-03-08T06:05:11.978417+0000","last_became_active":"2024-03-08T06:05:10.997630+0000","last_became_peered":"2024-03-08T06:05:10.997630+0000","last_unstale":"2024-03-08T06:05:11.978417+0000","last_undegraded":"2024-03-08T06:05:11.978417+0000","last_fullsized":"2024-03-08T06:05:11.978417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:46:29.006617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.211077+0000","last_change":"2024-03-08T06:05:11.998569+0000","last_active":"2024-03-08T06:05:13.211077+0000","last_peered":"2024-03-08T06:05:13.211077+0000","last_clean":"2024-03-08T06:05:13.211077+0000","last_became_active":"2024-03-08T06:05:11.998245+0000","last_became_peered":"2024-03-08T06:05:11.998245+0000","last_unstale":"2024-03-08T06:05:13.211077+0000","last_undegraded":"2024-03-08T06:05:13.211077+0000","last_fullsized":"2024-03-08T06:05:13.211077+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:45:00.013512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896931+0000","last_change":"2024-03-08T06:05:10.998256+0000","last_active":"2024-03-08T06:05:13.896931+0000","last_peered":"2024-03-08T06:05:13.896931+0000","last_clean":"2024-03-08T06:05:13.896931+0000","last_became_active":"2024-03-08T06:05:10.998010+0000","last_became_peered":"2024-03-08T06:05:10.998010+0000","last_unstale":"2024-03-08T06:05:13.896931+0000","last_undegraded":"2024-03-08T06:05:13.896931+0000","last_fullsized":"2024-03-08T06:05:13.896931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:48:48.169538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.009012+0000","last_change":"2024-03-08T06:05:12.005181+0000","last_active":"2024-03-08T06:05:12.009012+0000","last_peered":"2024-03-08T06:05:12.009012+0000","last_clean":"2024-03-08T06:05:12.009012+0000","last_became_active":"2024-03-08T06:05:12.004760+0000","last_became_peered":"2024-03-08T06:05:12.004760+0000","last_unstale":"2024-03-08T06:05:12.009012+0000","last_undegraded":"2024-03-08T06:05:12.009012+0000","last_fullsized":"2024-03-08T06:05:12.009012+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:57:44.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.711347+0000","last_change":"2024-03-08T06:05:10.984406+0000","last_active":"2024-03-08T06:05:13.711347+0000","last_peered":"2024-03-08T06:05:13.711347+0000","last_clean":"2024-03-08T06:05:13.711347+0000","last_became_active":"2024-03-08T06:05:10.984009+0000","last_became_peered":"2024-03-08T06:05:10.984009+0000","last_unstale":"2024-03-08T06:05:13.711347+0000","last_undegraded":"2024-03-08T06:05:13.711347+0000","last_fullsized":"2024-03-08T06:05:13.711347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:46:19.325205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.710803+0000","last_change":"2024-03-08T06:05:11.995709+0000","last_active":"2024-03-08T06:05:13.710803+0000","last_peered":"2024-03-08T06:05:13.710803+0000","last_clean":"2024-03-08T06:05:13.710803+0000","last_became_active":"2024-03-08T06:05:11.995159+0000","last_became_peered":"2024-03-08T06:05:11.995159+0000","last_unstale":"2024-03-08T06:05:13.710803+0000","last_undegraded":"2024-03-08T06:05:13.710803+0000","last_fullsized":"2024-03-08T06:05:13.710803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:48.083808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896186+0000","last_change":"2024-03-08T06:05:10.988540+0000","last_active":"2024-03-08T06:05:13.896186+0000","last_peered":"2024-03-08T06:05:13.896186+0000","last_clean":"2024-03-08T06:05:13.896186+0000","last_became_active":"2024-03-08T06:05:10.988239+0000","last_became_peered":"2024-03-08T06:05:10.988239+0000","last_unstale":"2024-03-08T06:05:13.896186+0000","last_undegraded":"2024-03-08T06:05:13.896186+0000","last_fullsized":"2024-03-08T06:05:13.896186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:46:19.325205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.998593+0000","last_change":"2024-03-08T06:05:11.996961+0000","last_active":"2024-03-08T06:05:12.998593+0000","last_peered":"2024-03-08T06:05:12.998593+0000","last_clean":"2024-03-08T06:05:12.998593+0000","last_became_active":"2024-03-08T06:05:11.996003+0000","last_became_peered":"2024-03-08T06:05:11.996003+0000","last_unstale":"2024-03-08T06:05:12.998593+0000","last_undegraded":"2024-03-08T06:05:12.998593+0000","last_fullsized":"2024-03-08T06:05:12.998593+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:17:37.572684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.711095+0000","last_change":"2024-03-08T06:05:10.992120+0000","last_active":"2024-03-08T06:05:13.711095+0000","last_peered":"2024-03-08T06:05:13.711095+0000","last_clean":"2024-03-08T06:05:13.711095+0000","last_became_active":"2024-03-08T06:05:10.991787+0000","last_became_peered":"2024-03-08T06:05:10.991787+0000","last_unstale":"2024-03-08T06:05:13.711095+0000","last_undegraded":"2024-03-08T06:05:13.711095+0000","last_fullsized":"2024-03-08T06:05:13.711095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:06:50.323035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.710995+0000","last_change":"2024-03-08T06:05:12.001256+0000","last_active":"2024-03-08T06:05:13.710995+0000","last_peered":"2024-03-08T06:05:13.710995+0000","last_clean":"2024-03-08T06:05:13.710995+0000","last_became_active":"2024-03-08T06:05:12.000929+0000","last_became_peered":"2024-03-08T06:05:12.000929+0000","last_unstale":"2024-03-08T06:05:13.710995+0000","last_undegraded":"2024-03-08T06:05:13.710995+0000","last_fullsized":"2024-03-08T06:05:13.710995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:01:49.887776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895692+0000","last_change":"2024-03-08T06:05:10.995950+0000","last_active":"2024-03-08T06:05:13.895692+0000","last_peered":"2024-03-08T06:05:13.895692+0000","last_clean":"2024-03-08T06:05:13.895692+0000","last_became_active":"2024-03-08T06:05:10.995663+0000","last_became_peered":"2024-03-08T06:05:10.995663+0000","last_unstale":"2024-03-08T06:05:13.895692+0000","last_undegraded":"2024-03-08T06:05:13.895692+0000","last_fullsized":"2024-03-08T06:05:13.895692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:34:04.199497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036411+0000","last_change":"2024-03-08T06:05:11.999266+0000","last_active":"2024-03-08T06:05:14.036411+0000","last_peered":"2024-03-08T06:05:14.036411+0000","last_clean":"2024-03-08T06:05:14.036411+0000","last_became_active":"2024-03-08T06:05:11.998999+0000","last_became_peered":"2024-03-08T06:05:11.998999+0000","last_unstale":"2024-03-08T06:05:14.036411+0000","last_undegraded":"2024-03-08T06:05:14.036411+0000","last_fullsized":"2024-03-08T06:05:14.036411+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:23:46.040146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.986839+0000","last_change":"2024-03-08T06:05:10.991421+0000","last_active":"2024-03-08T06:05:11.986839+0000","last_peered":"2024-03-08T06:05:11.986839+0000","last_clean":"2024-03-08T06:05:11.986839+0000","last_became_active":"2024-03-08T06:05:10.991015+0000","last_became_peered":"2024-03-08T06:05:10.991015+0000","last_unstale":"2024-03-08T06:05:11.986839+0000","last_undegraded":"2024-03-08T06:05:11.986839+0000","last_fullsized":"2024-03-08T06:05:11.986839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:43.456309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.016983+0000","last_change":"2024-03-08T06:05:12.007461+0000","last_active":"2024-03-08T06:05:12.016983+0000","last_peered":"2024-03-08T06:05:12.016983+0000","last_clean":"2024-03-08T06:05:12.016983+0000","last_became_active":"2024-03-08T06:05:12.006646+0000","last_became_peered":"2024-03-08T06:05:12.006646+0000","last_unstale":"2024-03-08T06:05:12.016983+0000","last_undegraded":"2024-03-08T06:05:12.016983+0000","last_fullsized":"2024-03-08T06:05:12.016983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:51:00.406802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036752+0000","last_change":"2024-03-08T06:05:10.995496+0000","last_active":"2024-03-08T06:05:14.036752+0000","last_peered":"2024-03-08T06:05:14.036752+0000","last_clean":"2024-03-08T06:05:14.036752+0000","last_became_active":"2024-03-08T06:05:10.995211+0000","last_became_peered":"2024-03-08T06:05:10.995211+0000","last_unstale":"2024-03-08T06:05:14.036752+0000","last_undegraded":"2024-03-08T06:05:14.036752+0000","last_fullsized":"2024-03-08T06:05:14.036752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:07:22.656020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996290+0000","last_change":"2024-03-08T06:05:12.001885+0000","last_active":"2024-03-08T06:05:12.996290+0000","last_peered":"2024-03-08T06:05:12.996290+0000","last_clean":"2024-03-08T06:05:12.996290+0000","last_became_active":"2024-03-08T06:05:11.997821+0000","last_became_peered":"2024-03-08T06:05:11.997821+0000","last_unstale":"2024-03-08T06:05:12.996290+0000","last_undegraded":"2024-03-08T06:05:12.996290+0000","last_fullsized":"2024-03-08T06:05:12.996290+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:28:26.609404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.998614+0000","last_change":"2024-03-08T06:05:10.987378+0000","last_active":"2024-03-08T06:05:12.998614+0000","last_peered":"2024-03-08T06:05:12.998614+0000","last_clean":"2024-03-08T06:05:12.998614+0000","last_became_active":"2024-03-08T06:05:10.987097+0000","last_became_peered":"2024-03-08T06:05:10.987097+0000","last_unstale":"2024-03-08T06:05:12.998614+0000","last_undegraded":"2024-03-08T06:05:12.998614+0000","last_fullsized":"2024-03-08T06:05:12.998614+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:44:58.998250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":193,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996914+0000","last_change":"2024-03-08T06:03:57.473597+0000","last_active":"2024-03-08T06:05:12.996914+0000","last_peered":"2024-03-08T06:05:12.996914+0000","last_clean":"2024-03-08T06:05:12.996914+0000","last_became_active":"2024-03-08T06:03:57.468593+0000","last_became_peered":"2024-03-08T06:03:57.468593+0000","last_unstale":"2024-03-08T06:05:12.996914+0000","last_undegraded":"2024-03-08T06:05:12.996914+0000","last_fullsized":"2024-03-08T06:05:12.996914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_clean_scrub_stamp":"2024-03-08T06:03:56.214441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:45:28.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":983040,"data_stored":741745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":151903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":145091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":152921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":151357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":152921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":735355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:05:16.486 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:05:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.749+0000 7f196e000640 1 Processor -- start 2024-03-08T06:05:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.750+0000 7f196e000640 1 -- start start 2024-03-08T06:05:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.751+0000 7f196e000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f196815bea0 0x7f196815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.751+0000 7f196e000640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1968071df0 0x7f196806d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.751+0000 7f196e000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196815ceb0 con 0x7f1968145520 2024-03-08T06:05:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.751+0000 7f196e000640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196815c330 con 0x7f1968071df0 2024-03-08T06:05:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.751+0000 7f196e000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196815c610 con 0x7f196815bea0 2024-03-08T06:05:16.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.752+0000 7f196cc00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1968145520 0x7f19680716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43460/0 (socket says 172.21.15.40:43460) 2024-03-08T06:05:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.752+0000 7f196cc00640 1 -- 172.21.15.40:0/392883755 learned_addr learned my addr 172.21.15.40:0/392883755 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.752+0000 7f1967e00640 1 --2- 172.21.15.40:0/392883755 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f196815bea0 0x7f196815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.752+0000 7f196d600640 1 --2- 172.21.15.40:0/392883755 >> v2:172.21.15.53:3300/0 conn(0x7f1968071df0 0x7f196806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.752+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245423323 0 0) 0x7f196815ceb0 con 0x7f1968145520 2024-03-08T06:05:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.752+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1950003520 con 0x7f1968145520 2024-03-08T06:05:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 803036544 0 0) 0x7f1950003520 con 0x7f1968145520 2024-03-08T06:05:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f196815bea0 msgr2=0x7f196815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 --2- 172.21.15.40:0/392883755 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f196815bea0 0x7f196815d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 >> v2:172.21.15.53:3300/0 conn(0x7f1968071df0 msgr2=0x7f196806d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 --2- 172.21.15.40:0/392883755 >> v2:172.21.15.53:3300/0 conn(0x7f1968071df0 0x7f196806d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196815c8f0 con 0x7f1968145520 2024-03-08T06:05:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f195c001670 con 0x7f1968145520 2024-03-08T06:05:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 >> v1:172.21.15.40:6789/0 conn(0x7f1968145520 legacy=0x7f19680716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 --2- 172.21.15.40:0/392883755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19500039b0 0x7f1950003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 --2- 172.21.15.40:0/392883755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1950004800 0x7f1950004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 --2- 172.21.15.40:0/392883755 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19500051c0 0x7f195000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196815c330 con 0x7f1950004800 2024-03-08T06:05:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196815c610 con 0x7f19500039b0 2024-03-08T06:05:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19500045f0 con 0x7f19500051c0 2024-03-08T06:05:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.753+0000 7f1967e00640 1 --2- 172.21.15.40:0/392883755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1950004800 0x7f1950004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.754+0000 7f1967e00640 1 -- 172.21.15.40:0/392883755 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19500051c0 msgr2=0x7f195000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.754+0000 7f1967e00640 1 --2- 172.21.15.40:0/392883755 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19500051c0 0x7f195000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.754+0000 7f1967e00640 1 -- 172.21.15.40:0/392883755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19500039b0 msgr2=0x7f1950003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.754+0000 7f1967e00640 1 --2- 172.21.15.40:0/392883755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19500039b0 0x7f1950003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.754+0000 7f1967e00640 1 -- 172.21.15.40:0/392883755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1950003520 con 0x7f1950004800 2024-03-08T06:05:16.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.755+0000 7f1967e00640 1 --2- 172.21.15.40:0/392883755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1950004800 0x7f1950004c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1954009af0 tx=0x7f1950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f060b2f6fb1ca server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.755+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954039070 con 0x7f1950004800 2024-03-08T06:05:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.755+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195402c710 con 0x7f1950004800 2024-03-08T06:05:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.755+0000 7f1967400640 1 -- 172.21.15.40:0/392883755 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954009690 con 0x7f1950004800 2024-03-08T06:05:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.755+0000 7f196e000640 1 -- 172.21.15.40:0/392883755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1950004800 msgr2=0x7f1950004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.755+0000 7f196e000640 1 --2- 172.21.15.40:0/392883755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1950004800 0x7f1950004c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1954009af0 tx=0x7f1950005830 comp rx=0 tx=0).stop 2024-03-08T06:05:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.756+0000 7f196d600640 1 -- 172.21.15.40:0/392883755 reap_dead start 2024-03-08T06:05:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.756+0000 7f196e000640 1 -- 172.21.15.40:0/392883755 shutdown_connections 2024-03-08T06:05:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.756+0000 7f196e000640 1 -- 172.21.15.40:0/392883755 >> 172.21.15.40:0/392883755 conn(0x7f19680b1850 msgr2=0x7f19680b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.756+0000 7f196e000640 1 -- 172.21.15.40:0/392883755 shutdown_connections 2024-03-08T06:05:16.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.756+0000 7f196e000640 1 -- 172.21.15.40:0/392883755 wait complete. 2024-03-08T06:05:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.758+0000 7f196e000640 1 Processor -- start 2024-03-08T06:05:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.758+0000 7f196e000640 1 -- start start 2024-03-08T06:05:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.759+0000 7f196e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1950004800 0x7f196815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.759+0000 7f196e000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19681e6d10 0x7f196815d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.759+0000 7f196e000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19681e7500 0x7f19680b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.759+0000 7f196e000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19680af1c0 con 0x7f19681e7500 2024-03-08T06:05:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.759+0000 7f196e000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19680af040 con 0x7f1950004800 2024-03-08T06:05:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.759+0000 7f196e000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196806e410 con 0x7f19681e6d10 2024-03-08T06:05:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.759+0000 7f196d600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19681e7500 0x7f19680b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.759+0000 7f196d600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19681e7500 0x7f19680b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33636/0 (socket says 172.21.15.40:33636) 2024-03-08T06:05:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.759+0000 7f196d600640 1 -- 172.21.15.40:0/405044411 learned_addr learned my addr 172.21.15.40:0/405044411 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f196cc00640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1950004800 0x7f196815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f1967e00640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19681e6d10 0x7f196815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f196d600640 1 -- 172.21.15.40:0/405044411 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19681e6d10 msgr2=0x7f196815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f196d600640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19681e6d10 0x7f196815d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f196d600640 1 -- 172.21.15.40:0/405044411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1950004800 msgr2=0x7f196815b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f196d600640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1950004800 0x7f196815b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f196d600640 1 -- 172.21.15.40:0/405044411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196815dda0 con 0x7f19681e7500 2024-03-08T06:05:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f196cc00640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1950004800 0x7f196815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f196d600640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19681e7500 0x7f19680b2a70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f196000ede0 tx=0x7f196000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f1964c00640 1 -- 172.21.15.40:0/405044411 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960018020 con 0x7f19681e7500 2024-03-08T06:05:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f1964c00640 1 -- 172.21.15.40:0/405044411 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196000bd10 con 0x7f19681e7500 2024-03-08T06:05:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f1964c00640 1 -- 172.21.15.40:0/405044411 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960014640 con 0x7f19681e7500 2024-03-08T06:05:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.760+0000 7f196e000640 1 -- 172.21.15.40:0/405044411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19681e7d50 con 0x7f19681e7500 2024-03-08T06:05:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.761+0000 7f196e000640 1 -- 172.21.15.40:0/405044411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196815d060 con 0x7f19681e7500 2024-03-08T06:05:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.762+0000 7f196e000640 1 -- 172.21.15.40:0/405044411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1968075100 con 0x7f19681e7500 2024-03-08T06:05:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.763+0000 7f1964c00640 1 -- 172.21.15.40:0/405044411 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1960005d10 con 0x7f19681e7500 2024-03-08T06:05:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.764+0000 7f1964c00640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f194806b500 0x7f194806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.764+0000 7f1964c00640 1 -- 172.21.15.40:0/405044411 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f196008a8b0 con 0x7f19681e7500 2024-03-08T06:05:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.764+0000 7f196cc00640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f194806b500 0x7f194806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.765+0000 7f196cc00640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f194806b500 0x7f194806d9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f196815feb0 tx=0x7f195c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.769+0000 7f1964c00640 1 -- 172.21.15.40:0/405044411 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1960022030 con 0x7f19681e7500 2024-03-08T06:05:16.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.962+0000 7f196e000640 1 -- 172.21.15.40:0/405044411 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19680cdc60 con 0x7f194806b500 2024-03-08T06:05:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.981+0000 7f1964c00640 1 -- 172.21.15.40:0/405044411 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251932 (secure 0 0 0) 0x7f19680cdc60 con 0x7f194806b500 2024-03-08T06:05:16.986 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:05:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-03-08T06:05:16.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 -- 172.21.15.40:0/405044411 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f194806b500 msgr2=0x7f194806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f194806b500 0x7f194806d9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f196815feb0 tx=0x7f195c009040 comp rx=0 tx=0).stop 2024-03-08T06:05:16.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 -- 172.21.15.40:0/405044411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19681e7500 msgr2=0x7f19680b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:16.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19681e7500 0x7f19680b2a70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f196000ede0 tx=0x7f196000b480 comp rx=0 tx=0).stop 2024-03-08T06:05:16.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 -- 172.21.15.40:0/405044411 shutdown_connections 2024-03-08T06:05:16.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f194806b500 0x7f194806d9c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19681e7500 0x7f19680b2a70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19681e6d10 0x7f196815d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 --2- 172.21.15.40:0/405044411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1950004800 0x7f196815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:16.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 -- 172.21.15.40:0/405044411 >> 172.21.15.40:0/405044411 conn(0x7f19680b1850 msgr2=0x7f1968144840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:16.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 -- 172.21.15.40:0/405044411 shutdown_connections 2024-03-08T06:05:16.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:16.987+0000 7f196e000640 1 -- 172.21.15.40:0/405044411 wait complete. 2024-03-08T06:05:17.005 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-03-08T06:05:15.506638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237032,"kb_used_data":4196,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737688,"statfs":{"total":773094113280,"available":772851392512,"internally_reserved":0,"allocated":4296704,"data_stored":2376952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.563319"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.998822+0000","last_change":"2024-03-08T06:05:12.000209+0000","last_active":"2024-03-08T06:05:12.998822+0000","last_peered":"2024-03-08T06:05:12.998822+0000","last_clean":"2024-03-08T06:05:12.998822+0000","last_became_active":"2024-03-08T06:05:11.999852+0000","last_became_peered":"2024-03-08T06:05:11.999852+0000","last_unstale":"2024-03-08T06:05:12.998822+0000","last_undegraded":"2024-03-08T06:05:12.998822+0000","last_fullsized":"2024-03-08T06:05:12.998822+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:29:12.650016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037831+0000","last_change":"2024-03-08T06:05:12.011066+0000","last_active":"2024-03-08T06:05:14.037831+0000","last_peered":"2024-03-08T06:05:14.037831+0000","last_clean":"2024-03-08T06:05:14.037831+0000","last_became_active":"2024-03-08T06:05:12.010672+0000","last_became_peered":"2024-03-08T06:05:12.010672+0000","last_unstale":"2024-03-08T06:05:14.037831+0000","last_undegraded":"2024-03-08T06:05:14.037831+0000","last_fullsized":"2024-03-08T06:05:14.037831+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:33:43.934376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.009061+0000","last_change":"2024-03-08T06:05:12.004749+0000","last_active":"2024-03-08T06:05:12.009061+0000","last_peered":"2024-03-08T06:05:12.009061+0000","last_clean":"2024-03-08T06:05:12.009061+0000","last_became_active":"2024-03-08T06:05:12.004384+0000","last_became_peered":"2024-03-08T06:05:12.004384+0000","last_unstale":"2024-03-08T06:05:12.009061+0000","last_undegraded":"2024-03-08T06:05:12.009061+0000","last_fullsized":"2024-03-08T06:05:12.009061+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:05:46.243688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895788+0000","last_change":"2024-03-08T06:05:12.001581+0000","last_active":"2024-03-08T06:05:13.895788+0000","last_peered":"2024-03-08T06:05:13.895788+0000","last_clean":"2024-03-08T06:05:13.895788+0000","last_became_active":"2024-03-08T06:05:12.000954+0000","last_became_peered":"2024-03-08T06:05:12.000954+0000","last_unstale":"2024-03-08T06:05:13.895788+0000","last_undegraded":"2024-03-08T06:05:13.895788+0000","last_fullsized":"2024-03-08T06:05:13.895788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:05:46.243688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.210604+0000","last_change":"2024-03-08T06:05:11.997690+0000","last_active":"2024-03-08T06:05:13.210604+0000","last_peered":"2024-03-08T06:05:13.210604+0000","last_clean":"2024-03-08T06:05:13.210604+0000","last_became_active":"2024-03-08T06:05:11.992959+0000","last_became_peered":"2024-03-08T06:05:11.992959+0000","last_unstale":"2024-03-08T06:05:13.210604+0000","last_undegraded":"2024-03-08T06:05:13.210604+0000","last_fullsized":"2024-03-08T06:05:13.210604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:46.016337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037318+0000","last_change":"2024-03-08T06:05:12.017054+0000","last_active":"2024-03-08T06:05:14.037318+0000","last_peered":"2024-03-08T06:05:14.037318+0000","last_clean":"2024-03-08T06:05:14.037318+0000","last_became_active":"2024-03-08T06:05:12.014998+0000","last_became_peered":"2024-03-08T06:05:12.014998+0000","last_unstale":"2024-03-08T06:05:14.037318+0000","last_undegraded":"2024-03-08T06:05:14.037318+0000","last_fullsized":"2024-03-08T06:05:14.037318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:01:00.803269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.997189+0000","last_change":"2024-03-08T06:05:12.002105+0000","last_active":"2024-03-08T06:05:12.997189+0000","last_peered":"2024-03-08T06:05:12.997189+0000","last_clean":"2024-03-08T06:05:12.997189+0000","last_became_active":"2024-03-08T06:05:12.001435+0000","last_became_peered":"2024-03-08T06:05:12.001435+0000","last_unstale":"2024-03-08T06:05:12.997189+0000","last_undegraded":"2024-03-08T06:05:12.997189+0000","last_fullsized":"2024-03-08T06:05:12.997189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:01:00.803269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996682+0000","last_change":"2024-03-08T06:05:12.008201+0000","last_active":"2024-03-08T06:05:12.996682+0000","last_peered":"2024-03-08T06:05:12.996682+0000","last_clean":"2024-03-08T06:05:12.996682+0000","last_became_active":"2024-03-08T06:05:12.007548+0000","last_became_peered":"2024-03-08T06:05:12.007548+0000","last_unstale":"2024-03-08T06:05:12.996682+0000","last_undegraded":"2024-03-08T06:05:12.996682+0000","last_fullsized":"2024-03-08T06:05:12.996682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:01.683645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.017542+0000","last_change":"2024-03-08T06:05:12.014267+0000","last_active":"2024-03-08T06:05:12.017542+0000","last_peered":"2024-03-08T06:05:12.017542+0000","last_clean":"2024-03-08T06:05:12.017542+0000","last_became_active":"2024-03-08T06:05:12.013448+0000","last_became_peered":"2024-03-08T06:05:12.013448+0000","last_unstale":"2024-03-08T06:05:12.017542+0000","last_undegraded":"2024-03-08T06:05:12.017542+0000","last_fullsized":"2024-03-08T06:05:12.017542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:08:43.342583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896446+0000","last_change":"2024-03-08T06:05:12.023314+0000","last_active":"2024-03-08T06:05:13.896446+0000","last_peered":"2024-03-08T06:05:13.896446+0000","last_clean":"2024-03-08T06:05:13.896446+0000","last_became_active":"2024-03-08T06:05:12.022826+0000","last_became_peered":"2024-03-08T06:05:12.022826+0000","last_unstale":"2024-03-08T06:05:13.896446+0000","last_undegraded":"2024-03-08T06:05:13.896446+0000","last_fullsized":"2024-03-08T06:05:13.896446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:11:41.014727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.997452+0000","last_change":"2024-03-08T06:05:12.011238+0000","last_active":"2024-03-08T06:05:12.997452+0000","last_peered":"2024-03-08T06:05:12.997452+0000","last_clean":"2024-03-08T06:05:12.997452+0000","last_became_active":"2024-03-08T06:05:12.010914+0000","last_became_peered":"2024-03-08T06:05:12.010914+0000","last_unstale":"2024-03-08T06:05:12.997452+0000","last_undegraded":"2024-03-08T06:05:12.997452+0000","last_fullsized":"2024-03-08T06:05:12.997452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:13:01.505273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.018481+0000","last_change":"2024-03-08T06:05:12.016727+0000","last_active":"2024-03-08T06:05:12.018481+0000","last_peered":"2024-03-08T06:05:12.018481+0000","last_clean":"2024-03-08T06:05:12.018481+0000","last_became_active":"2024-03-08T06:05:12.016419+0000","last_became_peered":"2024-03-08T06:05:12.016419+0000","last_unstale":"2024-03-08T06:05:12.018481+0000","last_undegraded":"2024-03-08T06:05:12.018481+0000","last_fullsized":"2024-03-08T06:05:12.018481+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:28:26.609404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.210615+0000","last_change":"2024-03-08T06:05:11.997914+0000","last_active":"2024-03-08T06:05:13.210615+0000","last_peered":"2024-03-08T06:05:13.210615+0000","last_clean":"2024-03-08T06:05:13.210615+0000","last_became_active":"2024-03-08T06:05:11.994055+0000","last_became_peered":"2024-03-08T06:05:11.994055+0000","last_unstale":"2024-03-08T06:05:13.210615+0000","last_undegraded":"2024-03-08T06:05:13.210615+0000","last_fullsized":"2024-03-08T06:05:13.210615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:42:12.730065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037726+0000","last_change":"2024-03-08T06:05:11.998899+0000","last_active":"2024-03-08T06:05:14.037726+0000","last_peered":"2024-03-08T06:05:14.037726+0000","last_clean":"2024-03-08T06:05:14.037726+0000","last_became_active":"2024-03-08T06:05:11.998604+0000","last_became_peered":"2024-03-08T06:05:11.998604+0000","last_unstale":"2024-03-08T06:05:14.037726+0000","last_undegraded":"2024-03-08T06:05:14.037726+0000","last_fullsized":"2024-03-08T06:05:14.037726+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:41:55.777936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.008189+0000","last_change":"2024-03-08T06:05:12.003117+0000","last_active":"2024-03-08T06:05:12.008189+0000","last_peered":"2024-03-08T06:05:12.008189+0000","last_clean":"2024-03-08T06:05:12.008189+0000","last_became_active":"2024-03-08T06:05:12.002805+0000","last_became_peered":"2024-03-08T06:05:12.002805+0000","last_unstale":"2024-03-08T06:05:12.008189+0000","last_undegraded":"2024-03-08T06:05:12.008189+0000","last_fullsized":"2024-03-08T06:05:12.008189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:16:58.567142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.997139+0000","last_change":"2024-03-08T06:05:12.008429+0000","last_active":"2024-03-08T06:05:12.997139+0000","last_peered":"2024-03-08T06:05:12.997139+0000","last_clean":"2024-03-08T06:05:12.997139+0000","last_became_active":"2024-03-08T06:05:12.007904+0000","last_became_peered":"2024-03-08T06:05:12.007904+0000","last_unstale":"2024-03-08T06:05:12.997139+0000","last_undegraded":"2024-03-08T06:05:12.997139+0000","last_fullsized":"2024-03-08T06:05:12.997139+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:38:39.993958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.997731+0000","last_change":"2024-03-08T06:05:12.006193+0000","last_active":"2024-03-08T06:05:12.997731+0000","last_peered":"2024-03-08T06:05:12.997731+0000","last_clean":"2024-03-08T06:05:12.997731+0000","last_became_active":"2024-03-08T06:05:12.004837+0000","last_became_peered":"2024-03-08T06:05:12.004837+0000","last_unstale":"2024-03-08T06:05:12.997731+0000","last_undegraded":"2024-03-08T06:05:12.997731+0000","last_fullsized":"2024-03-08T06:05:12.997731+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:12:21.419108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037635+0000","last_change":"2024-03-08T06:05:12.010653+0000","last_active":"2024-03-08T06:05:14.037635+0000","last_peered":"2024-03-08T06:05:14.037635+0000","last_clean":"2024-03-08T06:05:14.037635+0000","last_became_active":"2024-03-08T06:05:12.005042+0000","last_became_peered":"2024-03-08T06:05:12.005042+0000","last_unstale":"2024-03-08T06:05:14.037635+0000","last_undegraded":"2024-03-08T06:05:14.037635+0000","last_fullsized":"2024-03-08T06:05:14.037635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:12:21.419108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037548+0000","last_change":"2024-03-08T06:05:11.998689+0000","last_active":"2024-03-08T06:05:14.037548+0000","last_peered":"2024-03-08T06:05:14.037548+0000","last_clean":"2024-03-08T06:05:14.037548+0000","last_became_active":"2024-03-08T06:05:11.997155+0000","last_became_peered":"2024-03-08T06:05:11.997155+0000","last_unstale":"2024-03-08T06:05:14.037548+0000","last_undegraded":"2024-03-08T06:05:14.037548+0000","last_fullsized":"2024-03-08T06:05:14.037548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:13:01.505273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.211331+0000","last_change":"2024-03-08T06:05:11.989306+0000","last_active":"2024-03-08T06:05:13.211331+0000","last_peered":"2024-03-08T06:05:13.211331+0000","last_clean":"2024-03-08T06:05:13.211331+0000","last_became_active":"2024-03-08T06:05:11.989009+0000","last_became_peered":"2024-03-08T06:05:11.989009+0000","last_unstale":"2024-03-08T06:05:13.211331+0000","last_undegraded":"2024-03-08T06:05:13.211331+0000","last_fullsized":"2024-03-08T06:05:13.211331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:54:15.830517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.998836+0000","last_change":"2024-03-08T06:05:11.996110+0000","last_active":"2024-03-08T06:05:11.998836+0000","last_peered":"2024-03-08T06:05:11.998836+0000","last_clean":"2024-03-08T06:05:11.998836+0000","last_became_active":"2024-03-08T06:05:11.995775+0000","last_became_peered":"2024-03-08T06:05:11.995775+0000","last_unstale":"2024-03-08T06:05:11.998836+0000","last_undegraded":"2024-03-08T06:05:11.998836+0000","last_fullsized":"2024-03-08T06:05:11.998836+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:38:52.367265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.017100+0000","last_change":"2024-03-08T06:05:12.006289+0000","last_active":"2024-03-08T06:05:12.017100+0000","last_peered":"2024-03-08T06:05:12.017100+0000","last_clean":"2024-03-08T06:05:12.017100+0000","last_became_active":"2024-03-08T06:05:12.005925+0000","last_became_peered":"2024-03-08T06:05:12.005925+0000","last_unstale":"2024-03-08T06:05:12.017100+0000","last_undegraded":"2024-03-08T06:05:12.017100+0000","last_fullsized":"2024-03-08T06:05:12.017100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:46:20.340467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.710916+0000","last_change":"2024-03-08T06:05:12.007400+0000","last_active":"2024-03-08T06:05:13.710916+0000","last_peered":"2024-03-08T06:05:13.710916+0000","last_clean":"2024-03-08T06:05:13.710916+0000","last_became_active":"2024-03-08T06:05:12.007164+0000","last_became_peered":"2024-03-08T06:05:12.007164+0000","last_unstale":"2024-03-08T06:05:13.710916+0000","last_undegraded":"2024-03-08T06:05:13.710916+0000","last_fullsized":"2024-03-08T06:05:13.710916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:07:23.671282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.001545+0000","last_change":"2024-03-08T06:05:11.994250+0000","last_active":"2024-03-08T06:05:12.001545+0000","last_peered":"2024-03-08T06:05:12.001545+0000","last_clean":"2024-03-08T06:05:12.001545+0000","last_became_active":"2024-03-08T06:05:11.993847+0000","last_became_peered":"2024-03-08T06:05:11.993847+0000","last_unstale":"2024-03-08T06:05:12.001545+0000","last_undegraded":"2024-03-08T06:05:12.001545+0000","last_fullsized":"2024-03-08T06:05:12.001545+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:07:23.671282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896825+0000","last_change":"2024-03-08T06:05:12.009917+0000","last_active":"2024-03-08T06:05:13.896825+0000","last_peered":"2024-03-08T06:05:13.896825+0000","last_clean":"2024-03-08T06:05:13.896825+0000","last_became_active":"2024-03-08T06:05:12.009672+0000","last_became_peered":"2024-03-08T06:05:12.009672+0000","last_unstale":"2024-03-08T06:05:13.896825+0000","last_undegraded":"2024-03-08T06:05:13.896825+0000","last_fullsized":"2024-03-08T06:05:13.896825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:28:26.609404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037815+0000","last_change":"2024-03-08T06:05:12.010910+0000","last_active":"2024-03-08T06:05:14.037815+0000","last_peered":"2024-03-08T06:05:14.037815+0000","last_clean":"2024-03-08T06:05:14.037815+0000","last_became_active":"2024-03-08T06:05:12.006915+0000","last_became_peered":"2024-03-08T06:05:12.006915+0000","last_unstale":"2024-03-08T06:05:14.037815+0000","last_undegraded":"2024-03-08T06:05:14.037815+0000","last_fullsized":"2024-03-08T06:05:14.037815+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:14:01.815309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896408+0000","last_change":"2024-03-08T06:05:11.999823+0000","last_active":"2024-03-08T06:05:13.896408+0000","last_peered":"2024-03-08T06:05:13.896408+0000","last_clean":"2024-03-08T06:05:13.896408+0000","last_became_active":"2024-03-08T06:05:11.999131+0000","last_became_peered":"2024-03-08T06:05:11.999131+0000","last_unstale":"2024-03-08T06:05:13.896408+0000","last_undegraded":"2024-03-08T06:05:13.896408+0000","last_fullsized":"2024-03-08T06:05:13.896408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:19:47.069509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.001933+0000","last_change":"2024-03-08T06:05:11.996383+0000","last_active":"2024-03-08T06:05:12.001933+0000","last_peered":"2024-03-08T06:05:12.001933+0000","last_clean":"2024-03-08T06:05:12.001933+0000","last_became_active":"2024-03-08T06:05:11.996081+0000","last_became_peered":"2024-03-08T06:05:11.996081+0000","last_unstale":"2024-03-08T06:05:12.001933+0000","last_undegraded":"2024-03-08T06:05:12.001933+0000","last_fullsized":"2024-03-08T06:05:12.001933+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:01:49.887776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.015733+0000","last_change":"2024-03-08T06:05:12.005994+0000","last_active":"2024-03-08T06:05:12.015733+0000","last_peered":"2024-03-08T06:05:12.015733+0000","last_clean":"2024-03-08T06:05:12.015733+0000","last_became_active":"2024-03-08T06:05:12.005673+0000","last_became_peered":"2024-03-08T06:05:12.005673+0000","last_unstale":"2024-03-08T06:05:12.015733+0000","last_undegraded":"2024-03-08T06:05:12.015733+0000","last_fullsized":"2024-03-08T06:05:12.015733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:48:49.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996584+0000","last_change":"2024-03-08T06:05:12.001065+0000","last_active":"2024-03-08T06:05:12.996584+0000","last_peered":"2024-03-08T06:05:12.996584+0000","last_clean":"2024-03-08T06:05:12.996584+0000","last_became_active":"2024-03-08T06:05:12.000571+0000","last_became_peered":"2024-03-08T06:05:12.000571+0000","last_unstale":"2024-03-08T06:05:12.996584+0000","last_undegraded":"2024-03-08T06:05:12.996584+0000","last_fullsized":"2024-03-08T06:05:12.996584+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:17:38.904380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.998792+0000","last_change":"2024-03-08T06:05:11.990653+0000","last_active":"2024-03-08T06:05:12.998792+0000","last_peered":"2024-03-08T06:05:12.998792+0000","last_clean":"2024-03-08T06:05:12.998792+0000","last_became_active":"2024-03-08T06:05:11.990061+0000","last_became_peered":"2024-03-08T06:05:11.990061+0000","last_unstale":"2024-03-08T06:05:12.998792+0000","last_undegraded":"2024-03-08T06:05:12.998792+0000","last_fullsized":"2024-03-08T06:05:12.998792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:38:38.055804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.995966+0000","last_change":"2024-03-08T06:05:12.007166+0000","last_active":"2024-03-08T06:05:12.995966+0000","last_peered":"2024-03-08T06:05:12.995966+0000","last_clean":"2024-03-08T06:05:12.995966+0000","last_became_active":"2024-03-08T06:05:12.003713+0000","last_became_peered":"2024-03-08T06:05:12.003713+0000","last_unstale":"2024-03-08T06:05:12.995966+0000","last_undegraded":"2024-03-08T06:05:12.995966+0000","last_fullsized":"2024-03-08T06:05:12.995966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:21:16.418811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.711201+0000","last_change":"2024-03-08T06:05:12.007927+0000","last_active":"2024-03-08T06:05:13.711201+0000","last_peered":"2024-03-08T06:05:13.711201+0000","last_clean":"2024-03-08T06:05:13.711201+0000","last_became_active":"2024-03-08T06:05:12.007457+0000","last_became_peered":"2024-03-08T06:05:12.007457+0000","last_unstale":"2024-03-08T06:05:13.711201+0000","last_undegraded":"2024-03-08T06:05:13.711201+0000","last_fullsized":"2024-03-08T06:05:13.711201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:45:29.983658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.999163+0000","last_change":"2024-03-08T06:05:10.999998+0000","last_active":"2024-03-08T06:05:12.999163+0000","last_peered":"2024-03-08T06:05:12.999163+0000","last_clean":"2024-03-08T06:05:12.999163+0000","last_became_active":"2024-03-08T06:05:10.999712+0000","last_became_peered":"2024-03-08T06:05:10.999712+0000","last_unstale":"2024-03-08T06:05:12.999163+0000","last_undegraded":"2024-03-08T06:05:12.999163+0000","last_fullsized":"2024-03-08T06:05:12.999163+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:54:14.815255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.999159+0000","last_change":"2024-03-08T06:05:11.998105+0000","last_active":"2024-03-08T06:05:12.999159+0000","last_peered":"2024-03-08T06:05:12.999159+0000","last_clean":"2024-03-08T06:05:12.999159+0000","last_became_active":"2024-03-08T06:05:11.997785+0000","last_became_peered":"2024-03-08T06:05:11.997785+0000","last_unstale":"2024-03-08T06:05:12.999159+0000","last_undegraded":"2024-03-08T06:05:12.999159+0000","last_fullsized":"2024-03-08T06:05:12.999159+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:46:30.021879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.711315+0000","last_change":"2024-03-08T06:05:10.996010+0000","last_active":"2024-03-08T06:05:13.711315+0000","last_peered":"2024-03-08T06:05:13.711315+0000","last_clean":"2024-03-08T06:05:13.711315+0000","last_became_active":"2024-03-08T06:05:10.995723+0000","last_became_peered":"2024-03-08T06:05:10.995723+0000","last_unstale":"2024-03-08T06:05:13.711315+0000","last_undegraded":"2024-03-08T06:05:13.711315+0000","last_fullsized":"2024-03-08T06:05:13.711315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:48:48.169538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.015669+0000","last_change":"2024-03-08T06:05:12.005945+0000","last_active":"2024-03-08T06:05:12.015669+0000","last_peered":"2024-03-08T06:05:12.015669+0000","last_clean":"2024-03-08T06:05:12.015669+0000","last_became_active":"2024-03-08T06:05:12.005589+0000","last_became_peered":"2024-03-08T06:05:12.005589+0000","last_unstale":"2024-03-08T06:05:12.015669+0000","last_undegraded":"2024-03-08T06:05:12.015669+0000","last_fullsized":"2024-03-08T06:05:12.015669+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:06:51.338297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037331+0000","last_change":"2024-03-08T06:05:10.990927+0000","last_active":"2024-03-08T06:05:14.037331+0000","last_peered":"2024-03-08T06:05:14.037331+0000","last_clean":"2024-03-08T06:05:14.037331+0000","last_became_active":"2024-03-08T06:05:10.990580+0000","last_became_peered":"2024-03-08T06:05:10.990580+0000","last_unstale":"2024-03-08T06:05:14.037331+0000","last_undegraded":"2024-03-08T06:05:14.037331+0000","last_fullsized":"2024-03-08T06:05:14.037331+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:46:29.006617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896751+0000","last_change":"2024-03-08T06:05:12.002689+0000","last_active":"2024-03-08T06:05:13.896751+0000","last_peered":"2024-03-08T06:05:13.896751+0000","last_clean":"2024-03-08T06:05:13.896751+0000","last_became_active":"2024-03-08T06:05:11.999478+0000","last_became_peered":"2024-03-08T06:05:11.999478+0000","last_unstale":"2024-03-08T06:05:13.896751+0000","last_undegraded":"2024-03-08T06:05:13.896751+0000","last_fullsized":"2024-03-08T06:05:13.896751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:21:16.418811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.211272+0000","last_change":"2024-03-08T06:05:10.993573+0000","last_active":"2024-03-08T06:05:13.211272+0000","last_peered":"2024-03-08T06:05:13.211272+0000","last_clean":"2024-03-08T06:05:13.211272+0000","last_became_active":"2024-03-08T06:05:10.993235+0000","last_became_peered":"2024-03-08T06:05:10.993235+0000","last_unstale":"2024-03-08T06:05:13.211272+0000","last_undegraded":"2024-03-08T06:05:13.211272+0000","last_fullsized":"2024-03-08T06:05:13.211272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:42:51.826683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996702+0000","last_change":"2024-03-08T06:05:12.000803+0000","last_active":"2024-03-08T06:05:12.996702+0000","last_peered":"2024-03-08T06:05:12.996702+0000","last_clean":"2024-03-08T06:05:12.996702+0000","last_became_active":"2024-03-08T06:05:12.000044+0000","last_became_peered":"2024-03-08T06:05:12.000044+0000","last_unstale":"2024-03-08T06:05:12.996702+0000","last_undegraded":"2024-03-08T06:05:12.996702+0000","last_fullsized":"2024-03-08T06:05:12.996702+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:33:12.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.022729+0000","last_change":"2024-03-08T06:05:10.986207+0000","last_active":"2024-03-08T06:05:12.022729+0000","last_peered":"2024-03-08T06:05:12.022729+0000","last_clean":"2024-03-08T06:05:12.022729+0000","last_became_active":"2024-03-08T06:05:10.985748+0000","last_became_peered":"2024-03-08T06:05:10.985748+0000","last_unstale":"2024-03-08T06:05:12.022729+0000","last_undegraded":"2024-03-08T06:05:12.022729+0000","last_fullsized":"2024-03-08T06:05:12.022729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:13:49.466728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.011933+0000","last_change":"2024-03-08T06:05:12.007888+0000","last_active":"2024-03-08T06:05:12.011933+0000","last_peered":"2024-03-08T06:05:12.011933+0000","last_clean":"2024-03-08T06:05:12.011933+0000","last_became_active":"2024-03-08T06:05:12.007570+0000","last_became_peered":"2024-03-08T06:05:12.007570+0000","last_unstale":"2024-03-08T06:05:12.011933+0000","last_undegraded":"2024-03-08T06:05:12.011933+0000","last_fullsized":"2024-03-08T06:05:12.011933+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:19:47.069509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.022778+0000","last_change":"2024-03-08T06:05:10.978505+0000","last_active":"2024-03-08T06:05:12.022778+0000","last_peered":"2024-03-08T06:05:12.022778+0000","last_clean":"2024-03-08T06:05:12.022778+0000","last_became_active":"2024-03-08T06:05:10.978175+0000","last_became_peered":"2024-03-08T06:05:10.978175+0000","last_unstale":"2024-03-08T06:05:12.022778+0000","last_undegraded":"2024-03-08T06:05:12.022778+0000","last_fullsized":"2024-03-08T06:05:12.022778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:01:32.412422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.003404+0000","last_change":"2024-03-08T06:05:11.998586+0000","last_active":"2024-03-08T06:05:12.003404+0000","last_peered":"2024-03-08T06:05:12.003404+0000","last_clean":"2024-03-08T06:05:12.003404+0000","last_became_active":"2024-03-08T06:05:11.998259+0000","last_became_peered":"2024-03-08T06:05:11.998259+0000","last_unstale":"2024-03-08T06:05:12.003404+0000","last_undegraded":"2024-03-08T06:05:12.003404+0000","last_fullsized":"2024-03-08T06:05:12.003404+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:54:15.830517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036635+0000","last_change":"2024-03-08T06:05:10.982867+0000","last_active":"2024-03-08T06:05:14.036635+0000","last_peered":"2024-03-08T06:05:14.036635+0000","last_clean":"2024-03-08T06:05:14.036635+0000","last_became_active":"2024-03-08T06:05:10.979795+0000","last_became_peered":"2024-03-08T06:05:10.979795+0000","last_unstale":"2024-03-08T06:05:14.036635+0000","last_undegraded":"2024-03-08T06:05:14.036635+0000","last_fullsized":"2024-03-08T06:05:14.036635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:17:36.557421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895303+0000","last_change":"2024-03-08T06:05:12.002205+0000","last_active":"2024-03-08T06:05:13.895303+0000","last_peered":"2024-03-08T06:05:13.895303+0000","last_clean":"2024-03-08T06:05:13.895303+0000","last_became_active":"2024-03-08T06:05:12.001597+0000","last_became_peered":"2024-03-08T06:05:12.001597+0000","last_unstale":"2024-03-08T06:05:13.895303+0000","last_undegraded":"2024-03-08T06:05:13.895303+0000","last_fullsized":"2024-03-08T06:05:13.895303+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:45:29.983658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.982497+0000","last_change":"2024-03-08T06:05:10.982065+0000","last_active":"2024-03-08T06:05:11.982497+0000","last_peered":"2024-03-08T06:05:11.982497+0000","last_clean":"2024-03-08T06:05:11.982497+0000","last_became_active":"2024-03-08T06:05:10.981682+0000","last_became_peered":"2024-03-08T06:05:10.981682+0000","last_unstale":"2024-03-08T06:05:11.982497+0000","last_undegraded":"2024-03-08T06:05:11.982497+0000","last_fullsized":"2024-03-08T06:05:11.982497+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:48:45.001075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.018446+0000","last_change":"2024-03-08T06:05:12.016443+0000","last_active":"2024-03-08T06:05:12.018446+0000","last_peered":"2024-03-08T06:05:12.018446+0000","last_clean":"2024-03-08T06:05:12.018446+0000","last_became_active":"2024-03-08T06:05:12.016114+0000","last_became_peered":"2024-03-08T06:05:12.016114+0000","last_unstale":"2024-03-08T06:05:12.018446+0000","last_undegraded":"2024-03-08T06:05:12.018446+0000","last_fullsized":"2024-03-08T06:05:12.018446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:21:16.418811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037792+0000","last_change":"2024-03-08T06:05:10.994763+0000","last_active":"2024-03-08T06:05:14.037792+0000","last_peered":"2024-03-08T06:05:14.037792+0000","last_clean":"2024-03-08T06:05:14.037792+0000","last_became_active":"2024-03-08T06:05:10.994452+0000","last_became_peered":"2024-03-08T06:05:10.994452+0000","last_unstale":"2024-03-08T06:05:14.037792+0000","last_undegraded":"2024-03-08T06:05:14.037792+0000","last_fullsized":"2024-03-08T06:05:14.037792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:01:48.872514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037675+0000","last_change":"2024-03-08T06:05:11.999617+0000","last_active":"2024-03-08T06:05:14.037675+0000","last_peered":"2024-03-08T06:05:14.037675+0000","last_clean":"2024-03-08T06:05:14.037675+0000","last_became_active":"2024-03-08T06:05:11.998297+0000","last_became_peered":"2024-03-08T06:05:11.998297+0000","last_unstale":"2024-03-08T06:05:14.037675+0000","last_undegraded":"2024-03-08T06:05:14.037675+0000","last_fullsized":"2024-03-08T06:05:14.037675+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:50:05.691691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.711497+0000","last_change":"2024-03-08T06:05:10.986943+0000","last_active":"2024-03-08T06:05:13.711497+0000","last_peered":"2024-03-08T06:05:13.711497+0000","last_clean":"2024-03-08T06:05:13.711497+0000","last_became_active":"2024-03-08T06:05:10.985728+0000","last_became_peered":"2024-03-08T06:05:10.985728+0000","last_unstale":"2024-03-08T06:05:13.711497+0000","last_undegraded":"2024-03-08T06:05:13.711497+0000","last_fullsized":"2024-03-08T06:05:13.711497+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:50:59.391540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037204+0000","last_change":"2024-03-08T06:05:12.007778+0000","last_active":"2024-03-08T06:05:14.037204+0000","last_peered":"2024-03-08T06:05:14.037204+0000","last_clean":"2024-03-08T06:05:14.037204+0000","last_became_active":"2024-03-08T06:05:12.007420+0000","last_became_peered":"2024-03-08T06:05:12.007420+0000","last_unstale":"2024-03-08T06:05:14.037204+0000","last_undegraded":"2024-03-08T06:05:14.037204+0000","last_fullsized":"2024-03-08T06:05:14.037204+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:38:39.993958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.018619+0000","last_change":"2024-03-08T06:05:10.999021+0000","last_active":"2024-03-08T06:05:12.018619+0000","last_peered":"2024-03-08T06:05:12.018619+0000","last_clean":"2024-03-08T06:05:12.018619+0000","last_became_active":"2024-03-08T06:05:10.998545+0000","last_became_peered":"2024-03-08T06:05:10.998545+0000","last_unstale":"2024-03-08T06:05:12.018619+0000","last_undegraded":"2024-03-08T06:05:12.018619+0000","last_fullsized":"2024-03-08T06:05:12.018619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:47.068546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.015422+0000","last_change":"2024-03-08T06:05:12.011129+0000","last_active":"2024-03-08T06:05:12.015422+0000","last_peered":"2024-03-08T06:05:12.015422+0000","last_clean":"2024-03-08T06:05:12.015422+0000","last_became_active":"2024-03-08T06:05:12.010810+0000","last_became_peered":"2024-03-08T06:05:12.010810+0000","last_unstale":"2024-03-08T06:05:12.015422+0000","last_undegraded":"2024-03-08T06:05:12.015422+0000","last_fullsized":"2024-03-08T06:05:12.015422+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:11:41.014727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.037366+0000","last_change":"2024-03-08T06:05:10.991380+0000","last_active":"2024-03-08T06:05:14.037366+0000","last_peered":"2024-03-08T06:05:14.037366+0000","last_clean":"2024-03-08T06:05:14.037366+0000","last_became_active":"2024-03-08T06:05:10.991091+0000","last_became_peered":"2024-03-08T06:05:10.991091+0000","last_unstale":"2024-03-08T06:05:14.037366+0000","last_undegraded":"2024-03-08T06:05:14.037366+0000","last_fullsized":"2024-03-08T06:05:14.037366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:29:11.634754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895422+0000","last_change":"2024-03-08T06:05:12.000194+0000","last_active":"2024-03-08T06:05:13.895422+0000","last_peered":"2024-03-08T06:05:13.895422+0000","last_clean":"2024-03-08T06:05:13.895422+0000","last_became_active":"2024-03-08T06:05:11.999934+0000","last_became_peered":"2024-03-08T06:05:11.999934+0000","last_unstale":"2024-03-08T06:05:13.895422+0000","last_undegraded":"2024-03-08T06:05:13.895422+0000","last_fullsized":"2024-03-08T06:05:13.895422+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:16:58.567142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.017578+0000","last_change":"2024-03-08T06:05:10.984763+0000","last_active":"2024-03-08T06:05:12.017578+0000","last_peered":"2024-03-08T06:05:12.017578+0000","last_clean":"2024-03-08T06:05:12.017578+0000","last_became_active":"2024-03-08T06:05:10.983755+0000","last_became_peered":"2024-03-08T06:05:10.983755+0000","last_unstale":"2024-03-08T06:05:12.017578+0000","last_undegraded":"2024-03-08T06:05:12.017578+0000","last_fullsized":"2024-03-08T06:05:12.017578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:40:13.164263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036960+0000","last_change":"2024-03-08T06:05:11.994532+0000","last_active":"2024-03-08T06:05:14.036960+0000","last_peered":"2024-03-08T06:05:14.036960+0000","last_clean":"2024-03-08T06:05:14.036960+0000","last_became_active":"2024-03-08T06:05:11.994237+0000","last_became_peered":"2024-03-08T06:05:11.994237+0000","last_unstale":"2024-03-08T06:05:14.036960+0000","last_undegraded":"2024-03-08T06:05:14.036960+0000","last_fullsized":"2024-03-08T06:05:14.036960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:33:12.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895340+0000","last_change":"2024-03-08T06:05:10.989606+0000","last_active":"2024-03-08T06:05:13.895340+0000","last_peered":"2024-03-08T06:05:13.895340+0000","last_clean":"2024-03-08T06:05:13.895340+0000","last_became_active":"2024-03-08T06:05:10.989316+0000","last_became_peered":"2024-03-08T06:05:10.989316+0000","last_unstale":"2024-03-08T06:05:13.895340+0000","last_undegraded":"2024-03-08T06:05:13.895340+0000","last_fullsized":"2024-03-08T06:05:13.895340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:54:14.815255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996585+0000","last_change":"2024-03-08T06:05:12.003091+0000","last_active":"2024-03-08T06:05:12.996585+0000","last_peered":"2024-03-08T06:05:12.996585+0000","last_clean":"2024-03-08T06:05:12.996585+0000","last_became_active":"2024-03-08T06:05:12.002717+0000","last_became_peered":"2024-03-08T06:05:12.002717+0000","last_unstale":"2024-03-08T06:05:12.996585+0000","last_undegraded":"2024-03-08T06:05:12.996585+0000","last_fullsized":"2024-03-08T06:05:12.996585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:46:44.252156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036582+0000","last_change":"2024-03-08T06:05:10.994819+0000","last_active":"2024-03-08T06:05:14.036582+0000","last_peered":"2024-03-08T06:05:14.036582+0000","last_clean":"2024-03-08T06:05:14.036582+0000","last_became_active":"2024-03-08T06:05:10.994555+0000","last_became_peered":"2024-03-08T06:05:10.994555+0000","last_unstale":"2024-03-08T06:05:14.036582+0000","last_undegraded":"2024-03-08T06:05:14.036582+0000","last_fullsized":"2024-03-08T06:05:14.036582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:38:37.040542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.999155+0000","last_change":"2024-03-08T06:05:12.001069+0000","last_active":"2024-03-08T06:05:12.999155+0000","last_peered":"2024-03-08T06:05:12.999155+0000","last_clean":"2024-03-08T06:05:12.999155+0000","last_became_active":"2024-03-08T06:05:12.000761+0000","last_became_peered":"2024-03-08T06:05:12.000761+0000","last_unstale":"2024-03-08T06:05:12.999155+0000","last_undegraded":"2024-03-08T06:05:12.999155+0000","last_fullsized":"2024-03-08T06:05:12.999155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:48.083808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036586+0000","last_change":"2024-03-08T06:05:10.990848+0000","last_active":"2024-03-08T06:05:14.036586+0000","last_peered":"2024-03-08T06:05:14.036586+0000","last_clean":"2024-03-08T06:05:14.036586+0000","last_became_active":"2024-03-08T06:05:10.990537+0000","last_became_peered":"2024-03-08T06:05:10.990537+0000","last_unstale":"2024-03-08T06:05:14.036586+0000","last_undegraded":"2024-03-08T06:05:14.036586+0000","last_fullsized":"2024-03-08T06:05:14.036586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:47.068546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.015559+0000","last_change":"2024-03-08T06:05:12.004997+0000","last_active":"2024-03-08T06:05:12.015559+0000","last_peered":"2024-03-08T06:05:12.015559+0000","last_clean":"2024-03-08T06:05:12.015559+0000","last_became_active":"2024-03-08T06:05:12.004542+0000","last_became_peered":"2024-03-08T06:05:12.004542+0000","last_unstale":"2024-03-08T06:05:12.015559+0000","last_undegraded":"2024-03-08T06:05:12.015559+0000","last_fullsized":"2024-03-08T06:05:12.015559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:34:05.214759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896774+0000","last_change":"2024-03-08T06:05:10.986357+0000","last_active":"2024-03-08T06:05:13.896774+0000","last_peered":"2024-03-08T06:05:13.896774+0000","last_clean":"2024-03-08T06:05:13.896774+0000","last_became_active":"2024-03-08T06:05:10.986053+0000","last_became_peered":"2024-03-08T06:05:10.986053+0000","last_unstale":"2024-03-08T06:05:13.896774+0000","last_undegraded":"2024-03-08T06:05:13.896774+0000","last_fullsized":"2024-03-08T06:05:13.896774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:06:50.323035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.002475+0000","last_change":"2024-03-08T06:05:11.999467+0000","last_active":"2024-03-08T06:05:12.002475+0000","last_peered":"2024-03-08T06:05:12.002475+0000","last_clean":"2024-03-08T06:05:12.002475+0000","last_became_active":"2024-03-08T06:05:11.999191+0000","last_became_peered":"2024-03-08T06:05:11.999191+0000","last_unstale":"2024-03-08T06:05:12.002475+0000","last_undegraded":"2024-03-08T06:05:12.002475+0000","last_fullsized":"2024-03-08T06:05:12.002475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:45:29.983658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.017582+0000","last_change":"2024-03-08T06:05:10.987573+0000","last_active":"2024-03-08T06:05:12.017582+0000","last_peered":"2024-03-08T06:05:12.017582+0000","last_clean":"2024-03-08T06:05:12.017582+0000","last_became_active":"2024-03-08T06:05:10.985656+0000","last_became_peered":"2024-03-08T06:05:10.985656+0000","last_unstale":"2024-03-08T06:05:12.017582+0000","last_undegraded":"2024-03-08T06:05:12.017582+0000","last_fullsized":"2024-03-08T06:05:12.017582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:44:58.998250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895886+0000","last_change":"2024-03-08T06:05:12.001244+0000","last_active":"2024-03-08T06:05:13.895886+0000","last_peered":"2024-03-08T06:05:13.895886+0000","last_clean":"2024-03-08T06:05:13.895886+0000","last_became_active":"2024-03-08T06:05:12.000937+0000","last_became_peered":"2024-03-08T06:05:12.000937+0000","last_unstale":"2024-03-08T06:05:13.895886+0000","last_undegraded":"2024-03-08T06:05:13.895886+0000","last_fullsized":"2024-03-08T06:05:13.895886+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:57:44.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.017410+0000","last_change":"2024-03-08T06:05:10.980519+0000","last_active":"2024-03-08T06:05:12.017410+0000","last_peered":"2024-03-08T06:05:12.017410+0000","last_clean":"2024-03-08T06:05:12.017410+0000","last_became_active":"2024-03-08T06:05:10.980234+0000","last_became_peered":"2024-03-08T06:05:10.980234+0000","last_unstale":"2024-03-08T06:05:12.017410+0000","last_undegraded":"2024-03-08T06:05:12.017410+0000","last_fullsized":"2024-03-08T06:05:12.017410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:31:48.237166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.211054+0000","last_change":"2024-03-08T06:05:12.003229+0000","last_active":"2024-03-08T06:05:13.211054+0000","last_peered":"2024-03-08T06:05:13.211054+0000","last_clean":"2024-03-08T06:05:13.211054+0000","last_became_active":"2024-03-08T06:05:12.002830+0000","last_became_peered":"2024-03-08T06:05:12.002830+0000","last_unstale":"2024-03-08T06:05:13.211054+0000","last_undegraded":"2024-03-08T06:05:13.211054+0000","last_fullsized":"2024-03-08T06:05:13.211054+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:42:25.705238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.991981+0000","last_change":"2024-03-08T06:05:10.988057+0000","last_active":"2024-03-08T06:05:11.991981+0000","last_peered":"2024-03-08T06:05:11.991981+0000","last_clean":"2024-03-08T06:05:11.991981+0000","last_became_active":"2024-03-08T06:05:10.984660+0000","last_became_peered":"2024-03-08T06:05:10.984660+0000","last_unstale":"2024-03-08T06:05:11.991981+0000","last_undegraded":"2024-03-08T06:05:11.991981+0000","last_fullsized":"2024-03-08T06:05:11.991981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:42:24.689976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.210677+0000","last_change":"2024-03-08T06:05:11.991397+0000","last_active":"2024-03-08T06:05:13.210677+0000","last_peered":"2024-03-08T06:05:13.210677+0000","last_clean":"2024-03-08T06:05:13.210677+0000","last_became_active":"2024-03-08T06:05:11.991120+0000","last_became_peered":"2024-03-08T06:05:11.991120+0000","last_unstale":"2024-03-08T06:05:13.210677+0000","last_undegraded":"2024-03-08T06:05:13.210677+0000","last_fullsized":"2024-03-08T06:05:13.210677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:44.471571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.710776+0000","last_change":"2024-03-08T06:05:10.985889+0000","last_active":"2024-03-08T06:05:13.710776+0000","last_peered":"2024-03-08T06:05:13.710776+0000","last_clean":"2024-03-08T06:05:13.710776+0000","last_became_active":"2024-03-08T06:05:10.985583+0000","last_became_peered":"2024-03-08T06:05:10.985583+0000","last_unstale":"2024-03-08T06:05:13.710776+0000","last_undegraded":"2024-03-08T06:05:13.710776+0000","last_fullsized":"2024-03-08T06:05:13.710776+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:34:04.199497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.211158+0000","last_change":"2024-03-08T06:05:11.997394+0000","last_active":"2024-03-08T06:05:13.211158+0000","last_peered":"2024-03-08T06:05:13.211158+0000","last_clean":"2024-03-08T06:05:13.211158+0000","last_became_active":"2024-03-08T06:05:11.993710+0000","last_became_peered":"2024-03-08T06:05:11.993710+0000","last_unstale":"2024-03-08T06:05:13.211158+0000","last_undegraded":"2024-03-08T06:05:13.211158+0000","last_fullsized":"2024-03-08T06:05:13.211158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:38:52.367265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.998439+0000","last_change":"2024-03-08T06:05:10.985463+0000","last_active":"2024-03-08T06:05:12.998439+0000","last_peered":"2024-03-08T06:05:12.998439+0000","last_clean":"2024-03-08T06:05:12.998439+0000","last_became_active":"2024-03-08T06:05:10.985175+0000","last_became_peered":"2024-03-08T06:05:10.985175+0000","last_unstale":"2024-03-08T06:05:12.998439+0000","last_undegraded":"2024-03-08T06:05:12.998439+0000","last_fullsized":"2024-03-08T06:05:12.998439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:38:51.352003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.210751+0000","last_change":"2024-03-08T06:05:11.997488+0000","last_active":"2024-03-08T06:05:13.210751+0000","last_peered":"2024-03-08T06:05:13.210751+0000","last_clean":"2024-03-08T06:05:13.210751+0000","last_became_active":"2024-03-08T06:05:11.995546+0000","last_became_peered":"2024-03-08T06:05:11.995546+0000","last_unstale":"2024-03-08T06:05:13.210751+0000","last_undegraded":"2024-03-08T06:05:13.210751+0000","last_fullsized":"2024-03-08T06:05:13.210751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:31:49.252429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.986652+0000","last_change":"2024-03-08T06:05:10.987956+0000","last_active":"2024-03-08T06:05:11.986652+0000","last_peered":"2024-03-08T06:05:11.986652+0000","last_clean":"2024-03-08T06:05:11.986652+0000","last_became_active":"2024-03-08T06:05:10.987520+0000","last_became_peered":"2024-03-08T06:05:10.987520+0000","last_unstale":"2024-03-08T06:05:11.986652+0000","last_undegraded":"2024-03-08T06:05:11.986652+0000","last_fullsized":"2024-03-08T06:05:11.986652+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T06:42:11.714803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996706+0000","last_change":"2024-03-08T06:05:12.005301+0000","last_active":"2024-03-08T06:05:12.996706+0000","last_peered":"2024-03-08T06:05:12.996706+0000","last_clean":"2024-03-08T06:05:12.996706+0000","last_became_active":"2024-03-08T06:05:12.004898+0000","last_became_peered":"2024-03-08T06:05:12.004898+0000","last_unstale":"2024-03-08T06:05:12.996706+0000","last_undegraded":"2024-03-08T06:05:12.996706+0000","last_fullsized":"2024-03-08T06:05:12.996706+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:23:46.040146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.978417+0000","last_change":"2024-03-08T06:05:10.998091+0000","last_active":"2024-03-08T06:05:11.978417+0000","last_peered":"2024-03-08T06:05:11.978417+0000","last_clean":"2024-03-08T06:05:11.978417+0000","last_became_active":"2024-03-08T06:05:10.997630+0000","last_became_peered":"2024-03-08T06:05:10.997630+0000","last_unstale":"2024-03-08T06:05:11.978417+0000","last_undegraded":"2024-03-08T06:05:11.978417+0000","last_fullsized":"2024-03-08T06:05:11.978417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T07:46:29.006617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.211077+0000","last_change":"2024-03-08T06:05:11.998569+0000","last_active":"2024-03-08T06:05:13.211077+0000","last_peered":"2024-03-08T06:05:13.211077+0000","last_clean":"2024-03-08T06:05:13.211077+0000","last_became_active":"2024-03-08T06:05:11.998245+0000","last_became_peered":"2024-03-08T06:05:11.998245+0000","last_unstale":"2024-03-08T06:05:13.211077+0000","last_undegraded":"2024-03-08T06:05:13.211077+0000","last_fullsized":"2024-03-08T06:05:13.211077+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:45:00.013512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896931+0000","last_change":"2024-03-08T06:05:10.998256+0000","last_active":"2024-03-08T06:05:13.896931+0000","last_peered":"2024-03-08T06:05:13.896931+0000","last_clean":"2024-03-08T06:05:13.896931+0000","last_became_active":"2024-03-08T06:05:10.998010+0000","last_became_peered":"2024-03-08T06:05:10.998010+0000","last_unstale":"2024-03-08T06:05:13.896931+0000","last_undegraded":"2024-03-08T06:05:13.896931+0000","last_fullsized":"2024-03-08T06:05:13.896931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:48:48.169538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.009012+0000","last_change":"2024-03-08T06:05:12.005181+0000","last_active":"2024-03-08T06:05:12.009012+0000","last_peered":"2024-03-08T06:05:12.009012+0000","last_clean":"2024-03-08T06:05:12.009012+0000","last_became_active":"2024-03-08T06:05:12.004760+0000","last_became_peered":"2024-03-08T06:05:12.004760+0000","last_unstale":"2024-03-08T06:05:12.009012+0000","last_undegraded":"2024-03-08T06:05:12.009012+0000","last_fullsized":"2024-03-08T06:05:12.009012+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:57:44.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.711347+0000","last_change":"2024-03-08T06:05:10.984406+0000","last_active":"2024-03-08T06:05:13.711347+0000","last_peered":"2024-03-08T06:05:13.711347+0000","last_clean":"2024-03-08T06:05:13.711347+0000","last_became_active":"2024-03-08T06:05:10.984009+0000","last_became_peered":"2024-03-08T06:05:10.984009+0000","last_unstale":"2024-03-08T06:05:13.711347+0000","last_undegraded":"2024-03-08T06:05:13.711347+0000","last_fullsized":"2024-03-08T06:05:13.711347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:46:19.325205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.710803+0000","last_change":"2024-03-08T06:05:11.995709+0000","last_active":"2024-03-08T06:05:13.710803+0000","last_peered":"2024-03-08T06:05:13.710803+0000","last_clean":"2024-03-08T06:05:13.710803+0000","last_became_active":"2024-03-08T06:05:11.995159+0000","last_became_peered":"2024-03-08T06:05:11.995159+0000","last_unstale":"2024-03-08T06:05:13.710803+0000","last_undegraded":"2024-03-08T06:05:13.710803+0000","last_fullsized":"2024-03-08T06:05:13.710803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:07:48.083808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.896186+0000","last_change":"2024-03-08T06:05:10.988540+0000","last_active":"2024-03-08T06:05:13.896186+0000","last_peered":"2024-03-08T06:05:13.896186+0000","last_clean":"2024-03-08T06:05:13.896186+0000","last_became_active":"2024-03-08T06:05:10.988239+0000","last_became_peered":"2024-03-08T06:05:10.988239+0000","last_unstale":"2024-03-08T06:05:13.896186+0000","last_undegraded":"2024-03-08T06:05:13.896186+0000","last_fullsized":"2024-03-08T06:05:13.896186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:46:19.325205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.998593+0000","last_change":"2024-03-08T06:05:11.996961+0000","last_active":"2024-03-08T06:05:12.998593+0000","last_peered":"2024-03-08T06:05:12.998593+0000","last_clean":"2024-03-08T06:05:12.998593+0000","last_became_active":"2024-03-08T06:05:11.996003+0000","last_became_peered":"2024-03-08T06:05:11.996003+0000","last_unstale":"2024-03-08T06:05:12.998593+0000","last_undegraded":"2024-03-08T06:05:12.998593+0000","last_fullsized":"2024-03-08T06:05:12.998593+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:17:37.572684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.711095+0000","last_change":"2024-03-08T06:05:10.992120+0000","last_active":"2024-03-08T06:05:13.711095+0000","last_peered":"2024-03-08T06:05:13.711095+0000","last_clean":"2024-03-08T06:05:13.711095+0000","last_became_active":"2024-03-08T06:05:10.991787+0000","last_became_peered":"2024-03-08T06:05:10.991787+0000","last_unstale":"2024-03-08T06:05:13.711095+0000","last_undegraded":"2024-03-08T06:05:13.711095+0000","last_fullsized":"2024-03-08T06:05:13.711095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:06:50.323035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.710995+0000","last_change":"2024-03-08T06:05:12.001256+0000","last_active":"2024-03-08T06:05:13.710995+0000","last_peered":"2024-03-08T06:05:13.710995+0000","last_clean":"2024-03-08T06:05:13.710995+0000","last_became_active":"2024-03-08T06:05:12.000929+0000","last_became_peered":"2024-03-08T06:05:12.000929+0000","last_unstale":"2024-03-08T06:05:13.710995+0000","last_undegraded":"2024-03-08T06:05:13.710995+0000","last_fullsized":"2024-03-08T06:05:13.710995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:01:49.887776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:13.895692+0000","last_change":"2024-03-08T06:05:10.995950+0000","last_active":"2024-03-08T06:05:13.895692+0000","last_peered":"2024-03-08T06:05:13.895692+0000","last_clean":"2024-03-08T06:05:13.895692+0000","last_became_active":"2024-03-08T06:05:10.995663+0000","last_became_peered":"2024-03-08T06:05:10.995663+0000","last_unstale":"2024-03-08T06:05:13.895692+0000","last_undegraded":"2024-03-08T06:05:13.895692+0000","last_fullsized":"2024-03-08T06:05:13.895692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:34:04.199497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036411+0000","last_change":"2024-03-08T06:05:11.999266+0000","last_active":"2024-03-08T06:05:14.036411+0000","last_peered":"2024-03-08T06:05:14.036411+0000","last_clean":"2024-03-08T06:05:14.036411+0000","last_became_active":"2024-03-08T06:05:11.998999+0000","last_became_peered":"2024-03-08T06:05:11.998999+0000","last_unstale":"2024-03-08T06:05:14.036411+0000","last_undegraded":"2024-03-08T06:05:14.036411+0000","last_fullsized":"2024-03-08T06:05:14.036411+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:23:46.040146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:11.986839+0000","last_change":"2024-03-08T06:05:10.991421+0000","last_active":"2024-03-08T06:05:11.986839+0000","last_peered":"2024-03-08T06:05:11.986839+0000","last_clean":"2024-03-08T06:05:11.986839+0000","last_became_active":"2024-03-08T06:05:10.991015+0000","last_became_peered":"2024-03-08T06:05:10.991015+0000","last_unstale":"2024-03-08T06:05:11.986839+0000","last_undegraded":"2024-03-08T06:05:11.986839+0000","last_fullsized":"2024-03-08T06:05:11.986839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:20:43.456309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.016983+0000","last_change":"2024-03-08T06:05:12.007461+0000","last_active":"2024-03-08T06:05:12.016983+0000","last_peered":"2024-03-08T06:05:12.016983+0000","last_clean":"2024-03-08T06:05:12.016983+0000","last_became_active":"2024-03-08T06:05:12.006646+0000","last_became_peered":"2024-03-08T06:05:12.006646+0000","last_unstale":"2024-03-08T06:05:12.016983+0000","last_undegraded":"2024-03-08T06:05:12.016983+0000","last_fullsized":"2024-03-08T06:05:12.016983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:51:00.406802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:14.036752+0000","last_change":"2024-03-08T06:05:10.995496+0000","last_active":"2024-03-08T06:05:14.036752+0000","last_peered":"2024-03-08T06:05:14.036752+0000","last_clean":"2024-03-08T06:05:14.036752+0000","last_became_active":"2024-03-08T06:05:10.995211+0000","last_became_peered":"2024-03-08T06:05:10.995211+0000","last_unstale":"2024-03-08T06:05:14.036752+0000","last_undegraded":"2024-03-08T06:05:14.036752+0000","last_fullsized":"2024-03-08T06:05:14.036752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:07:22.656020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996290+0000","last_change":"2024-03-08T06:05:12.001885+0000","last_active":"2024-03-08T06:05:12.996290+0000","last_peered":"2024-03-08T06:05:12.996290+0000","last_clean":"2024-03-08T06:05:12.996290+0000","last_became_active":"2024-03-08T06:05:11.997821+0000","last_became_peered":"2024-03-08T06:05:11.997821+0000","last_unstale":"2024-03-08T06:05:12.996290+0000","last_undegraded":"2024-03-08T06:05:12.996290+0000","last_fullsized":"2024-03-08T06:05:12.996290+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:10.942095+0000","last_clean_scrub_stamp":"2024-03-08T06:05:10.942095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:28:26.609404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.998614+0000","last_change":"2024-03-08T06:05:10.987378+0000","last_active":"2024-03-08T06:05:12.998614+0000","last_peered":"2024-03-08T06:05:12.998614+0000","last_clean":"2024-03-08T06:05:12.998614+0000","last_became_active":"2024-03-08T06:05:10.987097+0000","last_became_peered":"2024-03-08T06:05:10.987097+0000","last_unstale":"2024-03-08T06:05:12.998614+0000","last_undegraded":"2024-03-08T06:05:12.998614+0000","last_fullsized":"2024-03-08T06:05:12.998614+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:05:09.926833+0000","last_clean_scrub_stamp":"2024-03-08T06:05:09.926833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:44:58.998250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":193,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T06:05:12.996914+0000","last_change":"2024-03-08T06:03:57.473597+0000","last_active":"2024-03-08T06:05:12.996914+0000","last_peered":"2024-03-08T06:05:12.996914+0000","last_clean":"2024-03-08T06:05:12.996914+0000","last_became_active":"2024-03-08T06:03:57.468593+0000","last_became_peered":"2024-03-08T06:03:57.468593+0000","last_unstale":"2024-03-08T06:05:12.996914+0000","last_undegraded":"2024-03-08T06:05:12.996914+0000","last_fullsized":"2024-03-08T06:05:12.996914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_clean_scrub_stamp":"2024-03-08T06:03:56.214441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:45:28.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":983040,"data_stored":741745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":151903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":145091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":152921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":151357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":152921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":735355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:05:17.009 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-08T06:05:17.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-08T06:05:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.280+0000 7ff64d400640 1 Processor -- start 2024-03-08T06:05:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.280+0000 7ff64d400640 1 -- start start 2024-03-08T06:05:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.282+0000 7ff64d400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff6481455b0 0x7ff64815c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.282+0000 7ff64d400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6481604f0 0x7ff6480cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.282+0000 7ff64d400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6480cd5a0 con 0x7ff648071950 2024-03-08T06:05:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.282+0000 7ff64d400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6480cdc20 con 0x7ff6481455b0 2024-03-08T06:05:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.282+0000 7ff64d400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480ccfe0 con 0x7ff6481604f0 2024-03-08T06:05:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.282+0000 7ff646800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff6481455b0 0x7ff64815c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.282+0000 7ff646800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff6481455b0 0x7ff64815c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50188/0 (socket says 172.21.15.40:50188) 2024-03-08T06:05:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.282+0000 7ff646800640 1 -- 172.21.15.40:0/262000935 learned_addr learned my addr 172.21.15.40:0/262000935 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.283+0000 7ff645e00640 1 --2- 172.21.15.40:0/262000935 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6481604f0 0x7ff6480cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.283+0000 7ff646800640 1 -- 172.21.15.40:0/262000935 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6481604f0 msgr2=0x7ff6480cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.283+0000 7ff646800640 1 --2- 172.21.15.40:0/262000935 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6481604f0 0x7ff6480cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.283+0000 7ff646800640 1 -- 172.21.15.40:0/262000935 >> v1:172.21.15.53:6789/0 conn(0x7ff648071950 legacy=0x7ff6480ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.283+0000 7ff646800640 1 -- 172.21.15.40:0/262000935 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480cd2c0 con 0x7ff6481455b0 2024-03-08T06:05:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.283+0000 7ff645e00640 1 --2- 172.21.15.40:0/262000935 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6481604f0 0x7ff6480cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.284+0000 7ff646800640 1 --2- 172.21.15.40:0/262000935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6481455b0 0x7ff64815c1d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff6480cdda0 tx=0x7ff63802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a4722e1bf73742 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.284+0000 7ff645400640 1 -- 172.21.15.40:0/262000935 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638035020 con 0x7ff6481455b0 2024-03-08T06:05:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.284+0000 7ff645400640 1 -- 172.21.15.40:0/262000935 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff638031640 con 0x7ff6481455b0 2024-03-08T06:05:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.284+0000 7ff645400640 1 -- 172.21.15.40:0/262000935 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63802d040 con 0x7ff6481455b0 2024-03-08T06:05:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.285+0000 7ff64d400640 1 -- 172.21.15.40:0/262000935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6481455b0 msgr2=0x7ff64815c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.285+0000 7ff64d400640 1 --2- 172.21.15.40:0/262000935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6481455b0 0x7ff64815c1d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff6480cdda0 tx=0x7ff63802b8f0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.285+0000 7ff64d400640 1 -- 172.21.15.40:0/262000935 shutdown_connections 2024-03-08T06:05:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.285+0000 7ff64d400640 1 --2- 172.21.15.40:0/262000935 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6481604f0 0x7ff6480cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.285+0000 7ff64d400640 1 --2- 172.21.15.40:0/262000935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6481455b0 0x7ff64815c1d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.285+0000 7ff64d400640 1 -- 172.21.15.40:0/262000935 >> 172.21.15.40:0/262000935 conn(0x7ff64806ae30 msgr2=0x7ff64806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.285+0000 7ff64d400640 1 -- 172.21.15.40:0/262000935 shutdown_connections 2024-03-08T06:05:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.285+0000 7ff64d400640 1 -- 172.21.15.40:0/262000935 wait complete. 2024-03-08T06:05:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.286+0000 7ff64d400640 1 Processor -- start 2024-03-08T06:05:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.287+0000 7ff64d400640 1 -- start start 2024-03-08T06:05:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.287+0000 7ff64d400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648071950 0x7ff6481e5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.287+0000 7ff64d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6481455b0 0x7ff6481e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.287+0000 7ff64d400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6481604f0 0x7ff6480ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.287+0000 7ff64d400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648145990 con 0x7ff648071950 2024-03-08T06:05:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.287+0000 7ff64d400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64815c710 con 0x7ff6481455b0 2024-03-08T06:05:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.287+0000 7ff64d400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480ccbd0 con 0x7ff6481604f0 2024-03-08T06:05:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.288+0000 7ff646800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648071950 0x7ff6481e5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.288+0000 7ff646800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648071950 0x7ff6481e5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50190/0 (socket says 172.21.15.40:50190) 2024-03-08T06:05:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.288+0000 7ff646800640 1 -- 172.21.15.40:0/274981736 learned_addr learned my addr 172.21.15.40:0/274981736 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.288+0000 7ff647200640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6481604f0 0x7ff6480ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.288+0000 7ff645e00640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6481455b0 0x7ff6481e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.288+0000 7ff646800640 1 -- 172.21.15.40:0/274981736 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6481604f0 msgr2=0x7ff6480ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.288+0000 7ff646800640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6481604f0 0x7ff6480ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.288+0000 7ff646800640 1 -- 172.21.15.40:0/274981736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6481455b0 msgr2=0x7ff6481e81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.288+0000 7ff646800640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6481455b0 0x7ff6481e81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.288+0000 7ff646800640 1 -- 172.21.15.40:0/274981736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6481e86e0 con 0x7ff648071950 2024-03-08T06:05:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.289+0000 7ff646800640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648071950 0x7ff6481e5630 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff638000c00 tx=0x7ff6380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:17.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.289+0000 7ff63ea00640 1 -- 172.21.15.40:0/274981736 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638040070 con 0x7ff648071950 2024-03-08T06:05:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.289+0000 7ff63ea00640 1 -- 172.21.15.40:0/274981736 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff638039c10 con 0x7ff648071950 2024-03-08T06:05:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.289+0000 7ff64d400640 1 -- 172.21.15.40:0/274981736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6481e5c10 con 0x7ff648071950 2024-03-08T06:05:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.289+0000 7ff63ea00640 1 -- 172.21.15.40:0/274981736 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638035040 con 0x7ff648071950 2024-03-08T06:05:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.289+0000 7ff64d400640 1 -- 172.21.15.40:0/274981736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6481e64b0 con 0x7ff648071950 2024-03-08T06:05:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.290+0000 7ff64d400640 1 -- 172.21.15.40:0/274981736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff648067cb0 con 0x7ff648071950 2024-03-08T06:05:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.291+0000 7ff63ea00640 1 -- 172.21.15.40:0/274981736 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff63804b060 con 0x7ff648071950 2024-03-08T06:05:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.292+0000 7ff63ea00640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff62806b4d0 0x7ff62806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.292+0000 7ff63ea00640 1 -- 172.21.15.40:0/274981736 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6380abe00 con 0x7ff648071950 2024-03-08T06:05:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.292+0000 7ff645e00640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff62806b4d0 0x7ff62806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.293+0000 7ff645e00640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff62806b4d0 0x7ff62806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff6480cc550 tx=0x7ff634008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.297+0000 7ff63ea00640 1 -- 172.21.15.40:0/274981736 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff63802f300 con 0x7ff648071950 2024-03-08T06:05:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.570+0000 7ff64d400640 1 -- 172.21.15.40:0/274981736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff6480729c0 con 0x7ff648071950 2024-03-08T06:05:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.574+0000 7ff63ea00640 1 -- 172.21.15.40:0/274981736 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff63807abe0 con 0x7ff648071950 2024-03-08T06:05:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-03-08T06:05:17.575 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAWquplEsjANRAAl87CIlR0lmsGJba4MA4mdw== 2024-03-08T06:05:17.575 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-03-08T06:05:17.575 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-03-08T06:05:17.575 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-08T06:05:17.575 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-03-08T06:05:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 -- 172.21.15.40:0/274981736 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff62806b4d0 msgr2=0x7ff62806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff62806b4d0 0x7ff62806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff6480cc550 tx=0x7ff634008040 comp rx=0 tx=0).stop 2024-03-08T06:05:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 -- 172.21.15.40:0/274981736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648071950 msgr2=0x7ff6481e5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648071950 0x7ff6481e5630 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff638000c00 tx=0x7ff6380046c0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 -- 172.21.15.40:0/274981736 shutdown_connections 2024-03-08T06:05:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff62806b4d0 0x7ff62806d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff6481604f0 0x7ff6480ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6481455b0 0x7ff6481e81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 --2- 172.21.15.40:0/274981736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648071950 0x7ff6481e5630 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 -- 172.21.15.40:0/274981736 >> 172.21.15.40:0/274981736 conn(0x7ff64806ae30 msgr2=0x7ff648143cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 -- 172.21.15.40:0/274981736 shutdown_connections 2024-03-08T06:05:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.577+0000 7ff64d400640 1 -- 172.21.15.40:0/274981736 wait complete. 2024-03-08T06:05:17.597 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:05:17.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.816+0000 7fa951400640 1 Processor -- start 2024-03-08T06:05:17.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.816+0000 7fa951400640 1 -- start start 2024-03-08T06:05:17.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.818+0000 7fa951400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa94c145520 0x7fa94c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.818+0000 7fa951400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c15bea0 0x7fa94c0716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.818+0000 7fa951400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c15ceb0 con 0x7fa94c0af040 2024-03-08T06:05:17.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.818+0000 7fa951400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94c15c330 con 0x7fa94c145520 2024-03-08T06:05:17.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.818+0000 7fa951400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c15c610 con 0x7fa94c15bea0 2024-03-08T06:05:17.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.819+0000 7fa94b400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c15bea0 0x7fa94c0716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.819+0000 7fa94be00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa94c145520 0x7fa94c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.819+0000 7fa94b400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c15bea0 0x7fa94c0716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:50676/0 (socket says 172.21.15.40:50676) 2024-03-08T06:05:17.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.819+0000 7fa94b400640 1 -- 172.21.15.40:0/1591021384 learned_addr learned my addr 172.21.15.40:0/1591021384 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:17.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.819+0000 7fa94b400640 1 -- 172.21.15.40:0/1591021384 >> v2:172.21.15.53:3300/0 conn(0x7fa94c145520 msgr2=0x7fa94c15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:17.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.819+0000 7fa94b400640 1 --2- 172.21.15.40:0/1591021384 >> v2:172.21.15.53:3300/0 conn(0x7fa94c145520 0x7fa94c15d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.819+0000 7fa94b400640 1 -- 172.21.15.40:0/1591021384 >> v1:172.21.15.53:6789/0 conn(0x7fa94c0af040 legacy=0x7fa94c06d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:17.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.819+0000 7fa94b400640 1 -- 172.21.15.40:0/1591021384 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c15c8f0 con 0x7fa94c15bea0 2024-03-08T06:05:17.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.821+0000 7fa94b400640 1 --2- 172.21.15.40:0/1591021384 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c15bea0 0x7fa94c0716e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa938009870 tx=0x7fa93802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1da52820184769 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:17.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.822+0000 7fa94aa00640 1 -- 172.21.15.40:0/1591021384 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938034070 con 0x7fa94c15bea0 2024-03-08T06:05:17.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.822+0000 7fa94aa00640 1 -- 172.21.15.40:0/1591021384 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa938030840 con 0x7fa94c15bea0 2024-03-08T06:05:17.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.822+0000 7fa94aa00640 1 -- 172.21.15.40:0/1591021384 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938030c10 con 0x7fa94c15bea0 2024-03-08T06:05:17.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.823+0000 7fa951400640 1 -- 172.21.15.40:0/1591021384 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c15bea0 msgr2=0x7fa94c0716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:17.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.823+0000 7fa951400640 1 --2- 172.21.15.40:0/1591021384 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c15bea0 0x7fa94c0716e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa938009870 tx=0x7fa93802b980 comp rx=0 tx=0).stop 2024-03-08T06:05:17.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.823+0000 7fa951400640 1 -- 172.21.15.40:0/1591021384 shutdown_connections 2024-03-08T06:05:17.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.823+0000 7fa951400640 1 --2- 172.21.15.40:0/1591021384 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c15bea0 0x7fa94c0716e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.823+0000 7fa951400640 1 --2- 172.21.15.40:0/1591021384 >> v2:172.21.15.53:3300/0 conn(0x7fa94c145520 0x7fa94c15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.823+0000 7fa951400640 1 -- 172.21.15.40:0/1591021384 >> 172.21.15.40:0/1591021384 conn(0x7fa94c0b1850 msgr2=0x7fa94c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:17.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.823+0000 7fa951400640 1 -- 172.21.15.40:0/1591021384 shutdown_connections 2024-03-08T06:05:17.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.823+0000 7fa951400640 1 -- 172.21.15.40:0/1591021384 wait complete. 2024-03-08T06:05:17.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.825+0000 7fa951400640 1 Processor -- start 2024-03-08T06:05:17.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.825+0000 7fa951400640 1 -- start start 2024-03-08T06:05:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.826+0000 7fa951400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c0af040 0x7fa94c136530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.826+0000 7fa951400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c145520 0x7fa94c137790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa951400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa94c137cd0 0x7fa94c1386f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa951400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c06e590 con 0x7fa94c137cd0 2024-03-08T06:05:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa951400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c06e410 con 0x7fa94c0af040 2024-03-08T06:05:17.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa951400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c06e710 con 0x7fa94c145520 2024-03-08T06:05:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa950a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa94c137cd0 0x7fa94c1386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa950a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa94c137cd0 0x7fa94c1386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50194/0 (socket says 172.21.15.40:50194) 2024-03-08T06:05:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa950a00640 1 -- 172.21.15.40:0/474084146 learned_addr learned my addr 172.21.15.40:0/474084146 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa94b400640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c145520 0x7fa94c137790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa94be00640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c0af040 0x7fa94c136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa950a00640 1 -- 172.21.15.40:0/474084146 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c145520 msgr2=0x7fa94c137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa950a00640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c145520 0x7fa94c137790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa950a00640 1 -- 172.21.15.40:0/474084146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c0af040 msgr2=0x7fa94c136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa950a00640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c0af040 0x7fa94c136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.827+0000 7fa950a00640 1 -- 172.21.15.40:0/474084146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c136a70 con 0x7fa94c137cd0 2024-03-08T06:05:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.828+0000 7fa950a00640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa94c137cd0 0x7fa94c1386f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c7a0 tx=0x7fa94400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.828+0000 7fa943e00640 1 -- 172.21.15.40:0/474084146 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944003b70 con 0x7fa94c137cd0 2024-03-08T06:05:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.828+0000 7fa943e00640 1 -- 172.21.15.40:0/474084146 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa944003d30 con 0x7fa94c137cd0 2024-03-08T06:05:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.828+0000 7fa943e00640 1 -- 172.21.15.40:0/474084146 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94400aa90 con 0x7fa94c137cd0 2024-03-08T06:05:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.828+0000 7fa951400640 1 -- 172.21.15.40:0/474084146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c139190 con 0x7fa94c137cd0 2024-03-08T06:05:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.828+0000 7fa951400640 1 -- 172.21.15.40:0/474084146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c136cb0 con 0x7fa94c137cd0 2024-03-08T06:05:17.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.829+0000 7fa951400640 1 -- 172.21.15.40:0/474084146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c075100 con 0x7fa94c137cd0 2024-03-08T06:05:17.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.830+0000 7fa943e00640 1 -- 172.21.15.40:0/474084146 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa94400ac30 con 0x7fa94c137cd0 2024-03-08T06:05:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.831+0000 7fa943e00640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa92c06b4d0 0x7fa92c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.831+0000 7fa943e00640 1 -- 172.21.15.40:0/474084146 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa94408a1f0 con 0x7fa94c137cd0 2024-03-08T06:05:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.832+0000 7fa94be00640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa92c06b4d0 0x7fa92c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.832+0000 7fa94be00640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa92c06b4d0 0x7fa92c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009ad0 tx=0x7fa93c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:17.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:17.836+0000 7fa943e00640 1 -- 172.21.15.40:0/474084146 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa944058fd0 con 0x7fa94c137cd0 2024-03-08T06:05:18.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.074+0000 7fa951400640 1 -- 172.21.15.40:0/474084146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa94c13f950 con 0x7fa94c137cd0 2024-03-08T06:05:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.075+0000 7fa943e00640 1 -- 172.21.15.40:0/474084146 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4935 (secure 0 0 0) 0x7fa9440588d0 con 0x7fa94c137cd0 2024-03-08T06:05:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:05:18.076 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/2194146146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6835","nonce":2194146146},{"type":"v1","addr":"172.21.15.40:6837","nonce":2194146146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/831964898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":831964898},{"type":"v1","addr":"172.21.15.53:6835","nonce":831964898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4723,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2951334634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2951334634},{"type":"v1","addr":"172.21.15.53:6833","nonce":2951334634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:05:11.446139+0000","modified":"2024-03-08T06:05:14.485839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6836/1742457959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1742457959},{"type":"v1","addr":"172.21.15.40:6836","nonce":1742457959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:05:18.076 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2024-03-08T06:05:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 -- 172.21.15.40:0/474084146 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa92c06b4d0 msgr2=0x7fa92c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa92c06b4d0 0x7fa92c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009ad0 tx=0x7fa93c008040 comp rx=0 tx=0).stop 2024-03-08T06:05:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 -- 172.21.15.40:0/474084146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa94c137cd0 msgr2=0x7fa94c1386f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa94c137cd0 0x7fa94c1386f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c7a0 tx=0x7fa94400cc60 comp rx=0 tx=0).stop 2024-03-08T06:05:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 -- 172.21.15.40:0/474084146 shutdown_connections 2024-03-08T06:05:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa92c06b4d0 0x7fa92c06d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa94c137cd0 0x7fa94c1386f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa94c145520 0x7fa94c137790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 --2- 172.21.15.40:0/474084146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa94c0af040 0x7fa94c136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 -- 172.21.15.40:0/474084146 >> 172.21.15.40:0/474084146 conn(0x7fa94c0b1850 msgr2=0x7fa94c15fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 -- 172.21.15.40:0/474084146 shutdown_connections 2024-03-08T06:05:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.078+0000 7fa951400640 1 -- 172.21.15.40:0/474084146 wait complete. 2024-03-08T06:05:18.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-08T06:05:11.446139+0000', 'modified': '2024-03-08T06:05:14.485839+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5049}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5049': {'gid': 5049, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6836/1742457959', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 1742457959}, {'type': 'v1', 'addr': '172.21.15.40:6836', 'nonce': 1742457959}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-08T06:05:18.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-08T06:05:18.499 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-08T06:05:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.732+0000 7f570dc00640 1 Processor -- start 2024-03-08T06:05:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.732+0000 7f570dc00640 1 -- start start 2024-03-08T06:05:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.734+0000 7f570dc00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57081611f0 0x7f57080b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.734+0000 7f570dc00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57080d3410 con 0x7f57080b5bf0 2024-03-08T06:05:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.734+0000 7f570dc00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57080d36f0 con 0x7f57080d0ff0 2024-03-08T06:05:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.734+0000 7f570dc00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57080d1480 con 0x7f57081611f0 2024-03-08T06:05:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.734+0000 7f570d200640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f57080b5bf0 0x7f57080d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33992/0 (socket says 172.21.15.40:33992) 2024-03-08T06:05:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.734+0000 7f570d200640 1 -- 172.21.15.40:0/3389563771 learned_addr learned my addr 172.21.15.40:0/3389563771 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.734+0000 7f5707e00640 1 --2- 172.21.15.40:0/3389563771 >> v2:172.21.15.53:3300/0 conn(0x7f57081611f0 0x7f57080b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.734+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876114955 0 0) 0x7f57080d3410 con 0x7f57080b5bf0 2024-03-08T06:05:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.734+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56ec003520 con 0x7f57080b5bf0 2024-03-08T06:05:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3498125447 0 0) 0x7f56ec003520 con 0x7f57080b5bf0 2024-03-08T06:05:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 >> v2:172.21.15.53:3300/0 conn(0x7f57081611f0 msgr2=0x7f57080b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 --2- 172.21.15.40:0/3389563771 >> v2:172.21.15.53:3300/0 conn(0x7f57081611f0 0x7f57080b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 >> v1:172.21.15.53:6789/0 conn(0x7f57080d0ff0 legacy=0x7f57080b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57080ced10 con 0x7f57080b5bf0 2024-03-08T06:05:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f57000019f0 con 0x7f57080b5bf0 2024-03-08T06:05:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 >> v1:172.21.15.40:6789/0 conn(0x7f57080b5bf0 legacy=0x7f57080d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56ec003ae0 0x7f56ec003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ec004800 0x7f56ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 0x7f56ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec00c270 con 0x7f56ec0053c0 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec00c420 con 0x7f56ec004800 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec00c620 con 0x7f56ec003ae0 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.735+0000 7f570d200640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 0x7f56ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.736+0000 7f56ff400640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ec004800 0x7f56ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.736+0000 7f5707e00640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56ec003ae0 0x7f56ec003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.736+0000 7f570d200640 1 -- 172.21.15.40:0/3389563771 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56ec003ae0 msgr2=0x7f56ec003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.736+0000 7f570d200640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56ec003ae0 0x7f56ec003ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.736+0000 7f570d200640 1 -- 172.21.15.40:0/3389563771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ec004800 msgr2=0x7f56ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.736+0000 7f570d200640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ec004800 0x7f56ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.736+0000 7f570d200640 1 -- 172.21.15.40:0/3389563771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec003520 con 0x7f56ec0053c0 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.736+0000 7f56ff400640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ec004800 0x7f56ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:05:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.737+0000 7f570d200640 1 -- 172.21.15.40:0/3389563771 reap_dead start 2024-03-08T06:05:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.737+0000 7f570d200640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 0x7f56ec00b9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f570000a760 tx=0x7f56ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2641da74c229da server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.737+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570000a3b0 con 0x7f56ec0053c0 2024-03-08T06:05:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.737+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5700034650 con 0x7f56ec0053c0 2024-03-08T06:05:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.737+0000 7f5707400640 1 -- 172.21.15.40:0/3389563771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700034a40 con 0x7f56ec0053c0 2024-03-08T06:05:18.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.738+0000 7f570dc00640 1 -- 172.21.15.40:0/3389563771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 msgr2=0x7f56ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.738+0000 7f570dc00640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 0x7f56ec00b9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f570000a760 tx=0x7f56ec005830 comp rx=0 tx=0).stop 2024-03-08T06:05:18.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.738+0000 7f570dc00640 1 -- 172.21.15.40:0/3389563771 shutdown_connections 2024-03-08T06:05:18.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.738+0000 7f570dc00640 1 --2- 172.21.15.40:0/3389563771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 0x7f56ec00b9c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.738+0000 7f570dc00640 1 -- 172.21.15.40:0/3389563771 >> 172.21.15.40:0/3389563771 conn(0x7f570806c700 msgr2=0x7f570806cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:18.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.738+0000 7f570dc00640 1 -- 172.21.15.40:0/3389563771 shutdown_connections 2024-03-08T06:05:18.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.738+0000 7f570dc00640 1 -- 172.21.15.40:0/3389563771 wait complete. 2024-03-08T06:05:18.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.740+0000 7f570dc00640 1 Processor -- start 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.740+0000 7f570dc00640 1 -- start start 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.741+0000 7f570dc00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 0x7f57080cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.741+0000 7f570dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708150750 0x7f57080aa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.741+0000 7f570dc00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f570814d3e0 0x7f570814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.741+0000 7f570dc00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57080ceac0 con 0x7f56ec0053c0 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.741+0000 7f570dc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57080cf080 con 0x7f5708150750 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.741+0000 7f570dc00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57080b27a0 con 0x7f570814d3e0 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.741+0000 7f5707e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 0x7f57080cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.741+0000 7f5707e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 0x7f57080cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50208/0 (socket says 172.21.15.40:50208) 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.741+0000 7f5707e00640 1 -- 172.21.15.40:0/4290306334 learned_addr learned my addr 172.21.15.40:0/4290306334 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.741+0000 7f56ff400640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708150750 0x7f57080aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f5707e00640 1 -- 172.21.15.40:0/4290306334 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f570814d3e0 msgr2=0x7f570814fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f5707e00640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f570814d3e0 0x7f570814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f5707e00640 1 -- 172.21.15.40:0/4290306334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708150750 msgr2=0x7f57080aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f5707e00640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708150750 0x7f57080aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f5707e00640 1 -- 172.21.15.40:0/4290306334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570814da00 con 0x7f56ec0053c0 2024-03-08T06:05:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f56ff400640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708150750 0x7f57080aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f5707e00640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 0x7f57080cb170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f56f8011570 tx=0x7f56f8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f5704c00640 1 -- 172.21.15.40:0/4290306334 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8015930 con 0x7f56ec0053c0 2024-03-08T06:05:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f5704c00640 1 -- 172.21.15.40:0/4290306334 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f8015ad0 con 0x7f56ec0053c0 2024-03-08T06:05:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f5704c00640 1 -- 172.21.15.40:0/4290306334 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f801d400 con 0x7f56ec0053c0 2024-03-08T06:05:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f570dc00640 1 -- 172.21.15.40:0/4290306334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570814dce0 con 0x7f56ec0053c0 2024-03-08T06:05:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.742+0000 7f570dc00640 1 -- 172.21.15.40:0/4290306334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570814e2a0 con 0x7f56ec0053c0 2024-03-08T06:05:18.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.744+0000 7f5704c00640 1 -- 172.21.15.40:0/4290306334 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56f801d5a0 con 0x7f56ec0053c0 2024-03-08T06:05:18.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.745+0000 7f5704c00640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f56e406b4d0 0x7f56e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.745+0000 7f5704c00640 1 -- 172.21.15.40:0/4290306334 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56f808ab00 con 0x7f56ec0053c0 2024-03-08T06:05:18.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.746+0000 7f56ff400640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f56e406b4d0 0x7f56e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.746+0000 7f5711f7c740 1 Processor -- start 2024-03-08T06:05:18.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.746+0000 7f5711f7c740 1 -- start start 2024-03-08T06:05:18.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.747+0000 7f56ff400640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f56e406b4d0 0x7f56e406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57080ce900 tx=0x7f56f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:18.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.747+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f5b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.747+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f5e5d70 0x56398f5e8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.747+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5b8e70 0x56398f5e8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.747+0000 7f5711f7c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398f522600 con 0x56398f5b8e70 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.747+0000 7f5711f7c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398f4cb930 con 0x56398f5e5d70 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.747+0000 7f5711f7c740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x56398f51e6a0 con 0x56398f5e5990 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.747+0000 7f570d200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5b8e70 0x56398f5e8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.747+0000 7f570d200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5b8e70 0x56398f5e8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50220/0 (socket says 172.21.15.40:50220) 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.747+0000 7f570d200640 1 -- 172.21.15.40:0/2102735751 learned_addr learned my addr 172.21.15.40:0/2102735751 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.748+0000 7f5707e00640 1 --2- 172.21.15.40:0/2102735751 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f5b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.748+0000 7f56ff400640 1 --2- 172.21.15.40:0/2102735751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f5e5d70 0x56398f5e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.748+0000 7f570d200640 1 -- 172.21.15.40:0/2102735751 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 msgr2=0x56398f5b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.748+0000 7f570d200640 1 --2- 172.21.15.40:0/2102735751 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f5b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.748+0000 7f570d200640 1 -- 172.21.15.40:0/2102735751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f5e5d70 msgr2=0x56398f5e8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.748+0000 7f570d200640 1 --2- 172.21.15.40:0/2102735751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f5e5d70 0x56398f5e8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.748+0000 7f570d200640 1 -- 172.21.15.40:0/2102735751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56398f2372d0 con 0x56398f5b8e70 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.748+0000 7f570d200640 1 --2- 172.21.15.40:0/2102735751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5b8e70 0x56398f5e8c60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f570000a880 tx=0x7f570002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6b1f2ca7ec7fd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.748+0000 7f56fcc00640 1 -- 172.21.15.40:0/2102735751 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700039040 con 0x56398f5b8e70 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.748+0000 7f56fcc00640 1 -- 172.21.15.40:0/2102735751 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57000018f0 con 0x56398f5b8e70 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.749+0000 7f56fcc00640 1 -- 172.21.15.40:0/2102735751 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700004d80 con 0x56398f5b8e70 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.749+0000 7f5711f7c740 1 -- 172.21.15.40:0/2102735751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5b8e70 msgr2=0x56398f5e8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.749+0000 7f5711f7c740 1 --2- 172.21.15.40:0/2102735751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5b8e70 0x56398f5e8c60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f570000a880 tx=0x7f570002e000 comp rx=0 tx=0).stop 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.749+0000 7f5711f7c740 1 -- 172.21.15.40:0/2102735751 shutdown_connections 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.749+0000 7f5711f7c740 1 --2- 172.21.15.40:0/2102735751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5b8e70 0x56398f5e8c60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.749+0000 7f5711f7c740 1 --2- 172.21.15.40:0/2102735751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f5e5d70 0x56398f5e8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.749+0000 7f5711f7c740 1 --2- 172.21.15.40:0/2102735751 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f5b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.749+0000 7f5711f7c740 1 -- 172.21.15.40:0/2102735751 >> 172.21.15.40:0/2102735751 conn(0x56398f5baba0 msgr2=0x56398f603720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:18.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.749+0000 7f5711f7c740 1 -- 172.21.15.40:0/2102735751 shutdown_connections 2024-03-08T06:05:18.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.749+0000 7f5711f7c740 1 -- 172.21.15.40:0/2102735751 wait complete. 2024-03-08T06:05:18.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.750+0000 7f5711f7c740 1 -- start start 2024-03-08T06:05:18.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.750+0000 7f5711f7c740 1 Processor -- start 2024-03-08T06:05:18.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.751+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f616490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.751+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.751+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f613b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.751+0000 7f5711f7c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398f5bd140 con 0x56398f5e5d70 2024-03-08T06:05:18.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.751+0000 7f5711f7c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398f4ded90 con 0x56398f6136f0 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.751+0000 7f5711f7c740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x56398f5bc530 con 0x56398f5e5990 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.751+0000 7f5711f7c740 10 client.0 _ll_register_callbacks cb 0x56398f51a350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.752+0000 7f56ff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.752+0000 7f56ff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50230/0 (socket says 172.21.15.40:50230) 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.752+0000 7f56ff400640 1 -- 172.21.15.40:0/769989277 learned_addr learned my addr 172.21.15.40:0/769989277 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.752+0000 7f5707e00640 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f616490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.752+0000 7f570d200640 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f613b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.752+0000 7f56ff400640 1 -- 172.21.15.40:0/769989277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 msgr2=0x56398f616490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.752+0000 7f56ff400640 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f616490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.752+0000 7f56ff400640 1 -- 172.21.15.40:0/769989277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 msgr2=0x56398f613b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.752+0000 7f56ff400640 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f613b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.752+0000 7f56ff400640 1 -- 172.21.15.40:0/769989277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56398f3016b0 con 0x56398f5e5d70 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.753+0000 7f56ff400640 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fd5e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f56f0004960 tx=0x7f56f00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdff93bcb4b41c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.753+0000 7f56f5600640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.753+0000 7f56f5600640 1 -- 172.21.15.40:0/769989277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0010040 con 0x56398f5e5d70 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.753+0000 7f56f5600640 1 -- 172.21.15.40:0/769989277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56f0014d40 con 0x56398f5e5d70 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.753+0000 7f5711f7c740 1 -- 172.21.15.40:0/769989277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57000018f0 con 0x56398f5e5d70 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.753+0000 7f56f5600640 1 -- 172.21.15.40:0/769989277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56f0017420 con 0x56398f5e5d70 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.753+0000 7f5711f7c740 10 client.5226 fetch_fsmap learned FSMap version 16 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.753+0000 7f5711f7c740 1 -- 172.21.15.40:0/769989277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x56398f614580 con 0x56398f5e5d70 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.753+0000 7f56f5600640 1 -- 172.21.15.40:0/769989277 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57000018f0 con 0x56398f5e5d70 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.753+0000 7f56f5600640 1 -- 172.21.15.40:0/769989277 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56f002c050 con 0x56398f5e5d70 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.754+0000 7f5711f7c740 10 client.5226 fetch_fsmap finished waiting for FSMap version 16 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.754+0000 7f5711f7c740 10 client.5226 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-08T06:05:18.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.754+0000 7f5711f7c740 20 client.5226 populate_metadata read hostname 'smithi040' 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.754+0000 7f5711f7c740 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x56398f5ebce0 0x56398f621460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.754+0000 7f5711f7c740 4 client.5226 mds_command: new command op to 5049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.754+0000 7f5711f7c740 1 -- 172.21.15.40:0/769989277 --> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56398f52fb40 con 0x56398f5ebce0 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.754+0000 7f5707e00640 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x56398f5ebce0 0x56398f621460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.754+0000 7f5707e00640 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x56398f5ebce0 0x56398f621460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3165a6e74c01874 server_cookie=1ca302363140e648 in_seq=0 out_seq=0 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.754+0000 7f56f5600640 10 client.5226 ms_handle_connect on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f56f5600640 1 -- 172.21.15.40:0/769989277 <== mds.0 v2:172.21.15.40:6834/1742457959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56398f52fb40 con 0x56398f5ebce0 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f56f5600640 10 client.5226 handle_command_reply: tid=0 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f5711f7c740 1 client.5226 shutdown 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f5711f7c740 1 -- 172.21.15.40:0/769989277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 msgr2=0x56398f5fd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f5711f7c740 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fd5e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f56f0004960 tx=0x7f56f00036f0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f5711f7c740 1 -- 172.21.15.40:0/769989277 shutdown_connections 2024-03-08T06:05:18.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f5711f7c740 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x56398f5ebce0 0x56398f621460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f5711f7c740 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f613b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f5711f7c740 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fd5e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f5711f7c740 1 --2- 172.21.15.40:0/769989277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f616490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f56f5600640 0 client.5226 ms_handle_reset on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:18.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.755+0000 7f5711f7c740 1 -- 172.21.15.40:0/769989277 >> 172.21.15.40:0/769989277 conn(0x56398f5b8e70 msgr2=0x56398f604970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:18.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.756+0000 7f5711f7c740 1 -- 172.21.15.40:0/769989277 shutdown_connections 2024-03-08T06:05:18.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.756+0000 7f5711f7c740 1 -- 172.21.15.40:0/769989277 wait complete. 2024-03-08T06:05:18.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.756+0000 7f5711f7c740 20 client.5226 trim_cache size 0 max 16384 2024-03-08T06:05:18.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.773+0000 7f5711f7c740 1 Processor -- start 2024-03-08T06:05:18.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.773+0000 7f5711f7c740 1 -- start start 2024-03-08T06:05:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.774+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f5b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.774+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.774+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f6686d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.774+0000 7f5711f7c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57081ff420 con 0x56398f5e5d70 2024-03-08T06:05:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.774+0000 7f5711f7c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57080c8fd0 con 0x56398f6136f0 2024-03-08T06:05:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.774+0000 7f5711f7c740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570814f4f0 con 0x56398f5e5990 2024-03-08T06:05:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.774+0000 7f56ff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.774+0000 7f56ff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50236/0 (socket says 172.21.15.40:50236) 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.774+0000 7f56ff400640 1 -- 172.21.15.40:0/3103260843 learned_addr learned my addr 172.21.15.40:0/3103260843 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.774+0000 7f56ff400640 1 -- 172.21.15.40:0/3103260843 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 msgr2=0x56398f5b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.774+0000 7f5707e00640 1 --2- 172.21.15.40:0/3103260843 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f5b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.775+0000 7f56ff400640 1 --2- 172.21.15.40:0/3103260843 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f5b9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.775+0000 7f570d200640 1 --2- 172.21.15.40:0/3103260843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f6686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.775+0000 7f56ff400640 1 -- 172.21.15.40:0/3103260843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 msgr2=0x56398f6686d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.775+0000 7f56ff400640 1 --2- 172.21.15.40:0/3103260843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f6686d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.775+0000 7f56ff400640 1 -- 172.21.15.40:0/3103260843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56398f613ee0 con 0x56398f5e5d70 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.775+0000 7f56ff400640 1 --2- 172.21.15.40:0/3103260843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fec80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f56f0017780 tx=0x7f56f001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a867b4f33f95612c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.775+0000 7f56f7e00640 1 -- 172.21.15.40:0/3103260843 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0017820 con 0x56398f5e5d70 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.775+0000 7f56f7e00640 1 -- 172.21.15.40:0/3103260843 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f00179c0 con 0x56398f5e5d70 2024-03-08T06:05:18.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.775+0000 7f56f7e00640 1 -- 172.21.15.40:0/3103260843 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0017d70 con 0x56398f5e5d70 2024-03-08T06:05:18.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.776+0000 7f5711f7c740 1 -- 172.21.15.40:0/3103260843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 msgr2=0x56398f5fec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.776+0000 7f5711f7c740 1 --2- 172.21.15.40:0/3103260843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fec80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f56f0017780 tx=0x7f56f001d000 comp rx=0 tx=0).stop 2024-03-08T06:05:18.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.776+0000 7f5711f7c740 1 -- 172.21.15.40:0/3103260843 shutdown_connections 2024-03-08T06:05:18.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.776+0000 7f5711f7c740 1 --2- 172.21.15.40:0/3103260843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f6686d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.776+0000 7f5711f7c740 1 --2- 172.21.15.40:0/3103260843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5d70 0x56398f5fec80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.776+0000 7f5711f7c740 1 --2- 172.21.15.40:0/3103260843 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5990 0x56398f5b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.776+0000 7f5711f7c740 1 -- 172.21.15.40:0/3103260843 >> 172.21.15.40:0/3103260843 conn(0x56398f5b8e70 msgr2=0x56398f5b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:18.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.776+0000 7f5711f7c740 1 -- 172.21.15.40:0/3103260843 shutdown_connections 2024-03-08T06:05:18.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.776+0000 7f5711f7c740 1 -- 172.21.15.40:0/3103260843 wait complete. 2024-03-08T06:05:18.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.777+0000 7f5711f7c740 1 -- start start 2024-03-08T06:05:18.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.777+0000 7f5711f7c740 1 Processor -- start 2024-03-08T06:05:18.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5990 0x56398f615950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5d70 0x56398f665360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f5711f7c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f6658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f5711f7c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398f5ebb60 con 0x56398f5e5990 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f5711f7c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398f5fc5d0 con 0x56398f6136f0 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f5711f7c740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x56398f5ec160 con 0x56398f5e5d70 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f5711f7c740 10 client.0 _ll_register_callbacks cb 0x7f56ec0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f5707e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5990 0x56398f615950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f5707e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5990 0x56398f615950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50246/0 (socket says 172.21.15.40:50246) 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f5707e00640 1 -- 172.21.15.40:0/87102525 learned_addr learned my addr 172.21.15.40:0/87102525 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f570d200640 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f6658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f56ff400640 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5d70 0x56398f665360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.778+0000 7f5707e00640 1 -- 172.21.15.40:0/87102525 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5d70 msgr2=0x56398f665360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f5707e00640 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5d70 0x56398f665360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f5707e00640 1 -- 172.21.15.40:0/87102525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 msgr2=0x56398f6658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f5707e00640 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f6658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f5707e00640 1 -- 172.21.15.40:0/87102525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56398f61d570 con 0x56398f5e5990 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f570d200640 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f6658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f5707e00640 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5990 0x56398f615950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f56f8055640 tx=0x7f56f8058e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efa0956270eba67 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f56f5600640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f56f5600640 1 -- 172.21.15.40:0/87102525 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8075060 con 0x56398f5e5990 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f56f5600640 1 -- 172.21.15.40:0/87102525 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56f8055850 con 0x56398f5e5990 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f5711f7c740 1 -- 172.21.15.40:0/87102525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56f00179c0 con 0x56398f5e5990 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f56f5600640 1 -- 172.21.15.40:0/87102525 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56f8058630 con 0x56398f5e5990 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f5711f7c740 10 client.5232 fetch_fsmap learned FSMap version 16 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.779+0000 7f5711f7c740 1 -- 172.21.15.40:0/87102525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x56398f5e4440 con 0x56398f5e5990 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.780+0000 7f56f5600640 1 -- 172.21.15.40:0/87102525 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f00179c0 con 0x56398f5e5990 2024-03-08T06:05:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.780+0000 7f56f5600640 1 -- 172.21.15.40:0/87102525 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56f8025030 con 0x56398f5e5990 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.780+0000 7f5711f7c740 10 client.5232 fetch_fsmap finished waiting for FSMap version 16 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.780+0000 7f5711f7c740 10 client.5232 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.780+0000 7f5711f7c740 20 client.5232 populate_metadata read hostname 'smithi040' 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.780+0000 7f5711f7c740 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x56398f6017e0 0x56398f601bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.780+0000 7f5711f7c740 4 client.5232 mds_command: new command op to 5049 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.780+0000 7f5711f7c740 1 -- 172.21.15.40:0/87102525 --> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56398f529e90 con 0x56398f6017e0 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.780+0000 7f56ff400640 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x56398f6017e0 0x56398f601bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f56ff400640 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x56398f6017e0 0x56398f601bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=694ec8c3c183c746 server_cookie=d69e2dfa7b43e68b in_seq=0 out_seq=0 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f56f5600640 10 client.5232 ms_handle_connect on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f56f5600640 1 -- 172.21.15.40:0/87102525 <== mds.0 v2:172.21.15.40:6834/1742457959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56398f529e90 con 0x56398f6017e0 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f56f5600640 10 client.5232 handle_command_reply: tid=0 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f5711f7c740 1 client.5232 shutdown 2024-03-08T06:05:18.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f5711f7c740 1 -- 172.21.15.40:0/87102525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5990 msgr2=0x56398f615950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f5711f7c740 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5990 0x56398f615950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f56f8055640 tx=0x7f56f8058e00 comp rx=0 tx=0).stop 2024-03-08T06:05:18.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f5711f7c740 1 -- 172.21.15.40:0/87102525 shutdown_connections 2024-03-08T06:05:18.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f5711f7c740 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x56398f6017e0 0x56398f601bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f5711f7c740 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56398f6136f0 0x56398f6658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f5711f7c740 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56398f5e5d70 0x56398f665360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f5711f7c740 1 --2- 172.21.15.40:0/87102525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56398f5e5990 0x56398f615950 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f56f5600640 0 client.5232 ms_handle_reset on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:18.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.781+0000 7f5711f7c740 1 -- 172.21.15.40:0/87102525 >> 172.21.15.40:0/87102525 conn(0x56398f5b8e70 msgr2=0x56398f5e8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:18.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f5711f7c740 1 -- 172.21.15.40:0/87102525 shutdown_connections 2024-03-08T06:05:18.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f5711f7c740 1 -- 172.21.15.40:0/87102525 wait complete. 2024-03-08T06:05:18.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f5711f7c740 20 client.5232 trim_cache size 0 max 16384 2024-03-08T06:05:18.787 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:05:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "c4e20909-a427-4e44-a3fb-93b068b9d1f6", 2024-03-08T06:05:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-03-08T06:05:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5049, 2024-03-08T06:05:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-03-08T06:05:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-03-08T06:05:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-03-08T06:05:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.8001611190000002, 2024-03-08T06:05:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 16, 2024-03-08T06:05:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 34, 2024-03-08T06:05:18.787 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 34, 2024-03-08T06:05:18.788 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 15.623626304 2024-03-08T06:05:18.788 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:05:18.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f570dc00640 1 -- 172.21.15.40:0/4290306334 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f56e406b4d0 msgr2=0x7f56e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f570dc00640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f56e406b4d0 0x7f56e406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57080ce900 tx=0x7f56f0009040 comp rx=0 tx=0).stop 2024-03-08T06:05:18.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f570dc00640 1 -- 172.21.15.40:0/4290306334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 msgr2=0x7f57080cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:18.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f570dc00640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 0x7f57080cb170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f56f8011570 tx=0x7f56f8011a30 comp rx=0 tx=0).stop 2024-03-08T06:05:18.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f570dc00640 1 -- 172.21.15.40:0/4290306334 shutdown_connections 2024-03-08T06:05:18.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f570dc00640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f56e406b4d0 0x7f56e406d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f570dc00640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f570814d3e0 0x7f570814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f570dc00640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5708150750 0x7f57080aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f570dc00640 1 --2- 172.21.15.40:0/4290306334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0053c0 0x7f57080cb170 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:18.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.782+0000 7f570dc00640 1 -- 172.21.15.40:0/4290306334 >> 172.21.15.40:0/4290306334 conn(0x7f570806c700 msgr2=0x7f57080d2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:18.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.783+0000 7f570dc00640 1 -- 172.21.15.40:0/4290306334 shutdown_connections 2024-03-08T06:05:18.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:18.783+0000 7f570dc00640 1 -- 172.21.15.40:0/4290306334 wait complete. 2024-03-08T06:05:18.799 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:05:19.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.011+0000 7fb43ca00640 1 Processor -- start 2024-03-08T06:05:19.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.011+0000 7fb43ca00640 1 -- start start 2024-03-08T06:05:19.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.012+0000 7fb43ca00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb438144840 0x7fb43806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.012+0000 7fb43ca00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb438071600 0x7fb438071a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.012+0000 7fb43ca00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb43806eb30 0x7fb43806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.012+0000 7fb43ca00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb438144c20 con 0x7fb438071600 2024-03-08T06:05:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.012+0000 7fb43ca00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43806f4a0 con 0x7fb438144840 2024-03-08T06:05:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.012+0000 7fb43ca00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380720a0 con 0x7fb43806eb30 2024-03-08T06:05:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.013+0000 7fb42d400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb438071600 0x7fb438071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.013+0000 7fb42d400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb438071600 0x7fb438071a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50250/0 (socket says 172.21.15.40:50250) 2024-03-08T06:05:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.013+0000 7fb42d400640 1 -- 172.21.15.40:0/1996868655 learned_addr learned my addr 172.21.15.40:0/1996868655 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.013+0000 7fb436800640 1 --2- 172.21.15.40:0/1996868655 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb43806eb30 0x7fb43806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.013+0000 7fb435e00640 1 --2- 172.21.15.40:0/1996868655 >> v2:172.21.15.53:3300/0 conn(0x7fb438144840 0x7fb43806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.013+0000 7fb42d400640 1 -- 172.21.15.40:0/1996868655 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb43806eb30 msgr2=0x7fb43806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.013+0000 7fb42d400640 1 --2- 172.21.15.40:0/1996868655 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb43806eb30 0x7fb43806ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.013+0000 7fb42d400640 1 -- 172.21.15.40:0/1996868655 >> v2:172.21.15.53:3300/0 conn(0x7fb438144840 msgr2=0x7fb43806e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.013+0000 7fb42d400640 1 --2- 172.21.15.40:0/1996868655 >> v2:172.21.15.53:3300/0 conn(0x7fb438144840 0x7fb43806e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.013+0000 7fb42d400640 1 -- 172.21.15.40:0/1996868655 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43806d810 con 0x7fb438071600 2024-03-08T06:05:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.013+0000 7fb435e00640 1 --2- 172.21.15.40:0/1996868655 >> v2:172.21.15.53:3300/0 conn(0x7fb438144840 0x7fb43806e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:19.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.014+0000 7fb42d400640 1 --2- 172.21.15.40:0/1996868655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb438071600 0x7fb438071a70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb4380b4cd0 tx=0x7fb42002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95255053354c68 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:19.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.014+0000 7fb435400640 1 -- 172.21.15.40:0/1996868655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420035020 con 0x7fb438071600 2024-03-08T06:05:19.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.015+0000 7fb435400640 1 -- 172.21.15.40:0/1996868655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb420031640 con 0x7fb438071600 2024-03-08T06:05:19.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.015+0000 7fb435400640 1 -- 172.21.15.40:0/1996868655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42002d040 con 0x7fb438071600 2024-03-08T06:05:19.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.015+0000 7fb43ca00640 1 -- 172.21.15.40:0/1996868655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb438071600 msgr2=0x7fb438071a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.015+0000 7fb43ca00640 1 --2- 172.21.15.40:0/1996868655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb438071600 0x7fb438071a70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb4380b4cd0 tx=0x7fb42002b8f0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.015+0000 7fb43ca00640 1 -- 172.21.15.40:0/1996868655 shutdown_connections 2024-03-08T06:05:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.015+0000 7fb43ca00640 1 --2- 172.21.15.40:0/1996868655 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb43806eb30 0x7fb43806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.015+0000 7fb43ca00640 1 --2- 172.21.15.40:0/1996868655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb438071600 0x7fb438071a70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.015+0000 7fb43ca00640 1 --2- 172.21.15.40:0/1996868655 >> v2:172.21.15.53:3300/0 conn(0x7fb438144840 0x7fb43806e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.015+0000 7fb43ca00640 1 -- 172.21.15.40:0/1996868655 >> 172.21.15.40:0/1996868655 conn(0x7fb43806ac50 msgr2=0x7fb43806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.015+0000 7fb43ca00640 1 -- 172.21.15.40:0/1996868655 shutdown_connections 2024-03-08T06:05:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.015+0000 7fb43ca00640 1 -- 172.21.15.40:0/1996868655 wait complete. 2024-03-08T06:05:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.017+0000 7fb43ca00640 1 Processor -- start 2024-03-08T06:05:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.017+0000 7fb43ca00640 1 -- start start 2024-03-08T06:05:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.018+0000 7fb43ca00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43806eb30 0x7fb43813b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.018+0000 7fb43ca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb438144840 0x7fb438158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.018+0000 7fb43ca00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb438159290 0x7fb4381f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.018+0000 7fb43ca00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb438145010 con 0x7fb43806eb30 2024-03-08T06:05:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.018+0000 7fb43ca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43806ddd0 con 0x7fb438144840 2024-03-08T06:05:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.018+0000 7fb43ca00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4380cc160 con 0x7fb438159290 2024-03-08T06:05:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.018+0000 7fb435e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43806eb30 0x7fb43813b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.018+0000 7fb435e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43806eb30 0x7fb43813b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50252/0 (socket says 172.21.15.40:50252) 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.018+0000 7fb435e00640 1 -- 172.21.15.40:0/2428329649 learned_addr learned my addr 172.21.15.40:0/2428329649 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.019+0000 7fb42d400640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb438144840 0x7fb438158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.019+0000 7fb436800640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb438159290 0x7fb4381f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.019+0000 7fb435e00640 1 -- 172.21.15.40:0/2428329649 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb438159290 msgr2=0x7fb4381f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.019+0000 7fb435e00640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb438159290 0x7fb4381f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.019+0000 7fb435e00640 1 -- 172.21.15.40:0/2428329649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb438144840 msgr2=0x7fb438158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.019+0000 7fb435e00640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb438144840 0x7fb438158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.019+0000 7fb435e00640 1 -- 172.21.15.40:0/2428329649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43813bb40 con 0x7fb43806eb30 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.019+0000 7fb436800640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb438159290 0x7fb4381f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.019+0000 7fb42d400640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb438144840 0x7fb438158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.019+0000 7fb435e00640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43806eb30 0x7fb43813b2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb42800c820 tx=0x7fb42800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.019+0000 7fb42ea00640 1 -- 172.21.15.40:0/2428329649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428015070 con 0x7fb43806eb30 2024-03-08T06:05:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.020+0000 7fb42ea00640 1 -- 172.21.15.40:0/2428329649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb428007a30 con 0x7fb43806eb30 2024-03-08T06:05:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.020+0000 7fb43ca00640 1 -- 172.21.15.40:0/2428329649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43813b830 con 0x7fb43806eb30 2024-03-08T06:05:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.020+0000 7fb42ea00640 1 -- 172.21.15.40:0/2428329649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428007de0 con 0x7fb43806eb30 2024-03-08T06:05:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.020+0000 7fb43ca00640 1 -- 172.21.15.40:0/2428329649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43813c0c0 con 0x7fb43806eb30 2024-03-08T06:05:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.021+0000 7fb43ca00640 1 -- 172.21.15.40:0/2428329649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb438067cb0 con 0x7fb43806eb30 2024-03-08T06:05:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.022+0000 7fb42ea00640 1 -- 172.21.15.40:0/2428329649 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb428018070 con 0x7fb43806eb30 2024-03-08T06:05:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.023+0000 7fb42ea00640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb41406b4d0 0x7fb41406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.023+0000 7fb42ea00640 1 -- 172.21.15.40:0/2428329649 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb42808a140 con 0x7fb43806eb30 2024-03-08T06:05:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.023+0000 7fb42d400640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb41406b4d0 0x7fb41406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:19.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.024+0000 7fb42d400640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb41406b4d0 0x7fb41406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb420036000 tx=0x7fb420008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:19.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.028+0000 7fb42ea00640 1 -- 172.21.15.40:0/2428329649 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb428058f20 con 0x7fb43806eb30 2024-03-08T06:05:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.269+0000 7fb43ca00640 1 -- 172.21.15.40:0/2428329649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb438141960 con 0x7fb43806eb30 2024-03-08T06:05:19.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.270+0000 7fb42ea00640 1 -- 172.21.15.40:0/2428329649 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb428058820 con 0x7fb43806eb30 2024-03-08T06:05:19.271 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T06:05:19.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 -- 172.21.15.40:0/2428329649 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb41406b4d0 msgr2=0x7fb41406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:19.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb41406b4d0 0x7fb41406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb420036000 tx=0x7fb420008030 comp rx=0 tx=0).stop 2024-03-08T06:05:19.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 -- 172.21.15.40:0/2428329649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43806eb30 msgr2=0x7fb43813b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:19.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43806eb30 0x7fb43813b2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb42800c820 tx=0x7fb42800cce0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 -- 172.21.15.40:0/2428329649 shutdown_connections 2024-03-08T06:05:19.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb41406b4d0 0x7fb41406d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb438159290 0x7fb4381f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb438144840 0x7fb438158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 --2- 172.21.15.40:0/2428329649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43806eb30 0x7fb43813b2f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:19.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 -- 172.21.15.40:0/2428329649 >> 172.21.15.40:0/2428329649 conn(0x7fb43806ac50 msgr2=0x7fb4380b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:19.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 -- 172.21.15.40:0/2428329649 shutdown_connections 2024-03-08T06:05:19.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:19.273+0000 7fb43ca00640 1 -- 172.21.15.40:0/2428329649 wait complete. 2024-03-08T06:05:19.294 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-08T06:05:19.295 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:05:19.295 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-03-08T06:05:19.295 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-08T06:05:19.295 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T06:05:19.295 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-08T06:05:19.295 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-08T06:05:19.295 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:05:19.295 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-08T06:05:19.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:19.296 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-03-08T06:05:19.352 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:05:19.352 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:05:19.352 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:05:19.352 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:19.352 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-03-08T06:05:19.352 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:19.352 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:05:19.352 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:05:19.352 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-03-08T06:05:19.352 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:05:19.352 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42446sec preferred_lft 42446sec 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:05:19.353 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 4a:78:ac:fc:f1:79 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::18e3:96ff:fe16:9a48/64 scope link 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether fa:59:73:2d:1a:29 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::f859:73ff:fe2d:1a29/64 scope link 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 4a:78:ac:fc:f1:79 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::4878:acff:fefc:f179/64 scope link 2024-03-08T06:05:19.354 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:19.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:19.356 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:05:19.452 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-03-08T06:05:19.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:19.457 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-03-08T06:05:19.515 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-03-08T06:05:19.515 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:05:19.515 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-08T06:05:19.515 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-08T06:05:19.515 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:05:19.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:19.516 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-03-08T06:05:19.516 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-03-08T06:05:19.516 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-08T06:05:19.516 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-08T06:05:19.516 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-08T06:05:19.516 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-03-08T06:05:19.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:19.711 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-03-08T06:05:19.765 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-08T06:05:19.765 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-08T06:05:19.765 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-08T06:05:19.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:19.766 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-03-08T06:05:19.865 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:05:19.865 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-08T06:05:19.923 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-08T06:05:19.923 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:05:19.923 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-08T06:05:19.970 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-08T06:05:19.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:19.970 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-08T06:05:20.159 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-08T06:05:20.159 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-08T06:05:20.159 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-03-08T06:05:20.183 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-08T06:05:20.183 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-08T06:05:20.183 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-03-08T06:05:20.184 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:05:20.184 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:05:20.209 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-08T06:05:20.209 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-08T06:05:20.209 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-03-08T06:05:20.209 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:05:20.209 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:05:20.209 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790" 2024-03-08T06:05:20.209 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790:/ 2024-03-08T06:05:20.209 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c4e20909-a427-4e44-a3fb-93b068b9d1f6" will pass to kernel 2024-03-08T06:05:20.211 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-08T06:05:20.211 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:05:20.211 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:05:20.211 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:05:20.211 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:05:20.215 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:05:20.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:20.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-08T06:05:20.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:20.304 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:05:20.379 INFO:teuthology.orchestra.run.smithi040.stdout:client4649 2024-03-08T06:05:20.379 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:05:20.379 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:05:20.435 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T06:05:20.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.659+0000 7f2120e00640 1 Processor -- start 2024-03-08T06:05:20.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.659+0000 7f2120e00640 1 -- start start 2024-03-08T06:05:20.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.661+0000 7f2120e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f211c0cf040 0x7f211c06e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:20.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.661+0000 7f2120e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f211c071f50 0x7f211c15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:20.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.661+0000 7f2120e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211c0b4bf0 con 0x7f211c0b46c0 2024-03-08T06:05:20.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.661+0000 7f2120e00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211c06f400 con 0x7f211c071f50 2024-03-08T06:05:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.661+0000 7f2120e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c06d730 con 0x7f211c0cf040 2024-03-08T06:05:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.661+0000 7f211a800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f211c071f50 0x7f211c15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.661+0000 7f211a800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f211c071f50 0x7f211c15ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50268/0 (socket says 172.21.15.40:50268) 2024-03-08T06:05:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.661+0000 7f211a800640 1 -- 172.21.15.40:0/3503858578 learned_addr learned my addr 172.21.15.40:0/3503858578 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.662+0000 7f2119e00640 1 --2- 172.21.15.40:0/3503858578 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f211c0cf040 0x7f211c06e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.662+0000 7f211a800640 1 -- 172.21.15.40:0/3503858578 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f211c0cf040 msgr2=0x7f211c06e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.662+0000 7f211a800640 1 --2- 172.21.15.40:0/3503858578 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f211c0cf040 0x7f211c06e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.662+0000 7f211a800640 1 -- 172.21.15.40:0/3503858578 >> v1:172.21.15.53:6789/0 conn(0x7f211c0b46c0 legacy=0x7f211c071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.662+0000 7f211a800640 1 -- 172.21.15.40:0/3503858578 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c06da10 con 0x7f211c071f50 2024-03-08T06:05:20.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.663+0000 7f211a800640 1 --2- 172.21.15.40:0/3503858578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c071f50 0x7f211c15ed50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2114009e50 tx=0x7f211402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c860accfe463c227 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:20.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.663+0000 7f2118a00640 1 -- 172.21.15.40:0/3503858578 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211402ca70 con 0x7f211c071f50 2024-03-08T06:05:20.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.663+0000 7f2118a00640 1 -- 172.21.15.40:0/3503858578 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211402cc10 con 0x7f211c071f50 2024-03-08T06:05:20.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.664+0000 7f2118a00640 1 -- 172.21.15.40:0/3503858578 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140343f0 con 0x7f211c071f50 2024-03-08T06:05:20.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.664+0000 7f2120e00640 1 -- 172.21.15.40:0/3503858578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c071f50 msgr2=0x7f211c15ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:20.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.664+0000 7f2120e00640 1 --2- 172.21.15.40:0/3503858578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c071f50 0x7f211c15ed50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2114009e50 tx=0x7f211402bed0 comp rx=0 tx=0).stop 2024-03-08T06:05:20.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.664+0000 7f2120e00640 1 -- 172.21.15.40:0/3503858578 shutdown_connections 2024-03-08T06:05:20.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.664+0000 7f2120e00640 1 --2- 172.21.15.40:0/3503858578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c071f50 0x7f211c15ed50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:20.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.664+0000 7f2120e00640 1 --2- 172.21.15.40:0/3503858578 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f211c0cf040 0x7f211c06e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:20.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.664+0000 7f2120e00640 1 -- 172.21.15.40:0/3503858578 >> 172.21.15.40:0/3503858578 conn(0x7f211c06ae30 msgr2=0x7f211c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:20.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.664+0000 7f2120e00640 1 -- 172.21.15.40:0/3503858578 shutdown_connections 2024-03-08T06:05:20.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.664+0000 7f2120e00640 1 -- 172.21.15.40:0/3503858578 wait complete. 2024-03-08T06:05:20.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.666+0000 7f2120e00640 1 Processor -- start 2024-03-08T06:05:20.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.666+0000 7f2120e00640 1 -- start start 2024-03-08T06:05:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.667+0000 7f2120e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f211c071f50 0x7f211c0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.667+0000 7f2120e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211c0b46c0 0x7f211c141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.667+0000 7f2120e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0cf040 0x7f211c1e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.667+0000 7f2120e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c146ab0 con 0x7f211c0cf040 2024-03-08T06:05:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.667+0000 7f2120e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c146930 con 0x7f211c0b46c0 2024-03-08T06:05:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.667+0000 7f2120e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c146c30 con 0x7f211c071f50 2024-03-08T06:05:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.667+0000 7f211a800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0cf040 0x7f211c1e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.667+0000 7f211a800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0cf040 0x7f211c1e98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50276/0 (socket says 172.21.15.40:50276) 2024-03-08T06:05:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.667+0000 7f211a800640 1 -- 172.21.15.40:0/1574319571 learned_addr learned my addr 172.21.15.40:0/1574319571 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.668+0000 7f2119400640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211c0b46c0 0x7f211c141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.668+0000 7f211a800640 1 -- 172.21.15.40:0/1574319571 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f211c071f50 msgr2=0x7f211c0b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.668+0000 7f211a800640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f211c071f50 0x7f211c0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.668+0000 7f211a800640 1 -- 172.21.15.40:0/1574319571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211c0b46c0 msgr2=0x7f211c141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.668+0000 7f211a800640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211c0b46c0 0x7f211c141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.668+0000 7f211a800640 1 -- 172.21.15.40:0/1574319571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c0afb00 con 0x7f211c0cf040 2024-03-08T06:05:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.668+0000 7f2119400640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211c0b46c0 0x7f211c141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.668+0000 7f211a800640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0cf040 0x7f211c1e98e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2114036040 tx=0x7f2114004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.668+0000 7f2112000640 1 -- 172.21.15.40:0/1574319571 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114043020 con 0x7f211c0cf040 2024-03-08T06:05:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.669+0000 7f2112000640 1 -- 172.21.15.40:0/1574319571 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211403e450 con 0x7f211c0cf040 2024-03-08T06:05:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.669+0000 7f2112000640 1 -- 172.21.15.40:0/1574319571 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114009510 con 0x7f211c0cf040 2024-03-08T06:05:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.669+0000 7f2120e00640 1 -- 172.21.15.40:0/1574319571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c0afde0 con 0x7f211c0cf040 2024-03-08T06:05:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.669+0000 7f2120e00640 1 -- 172.21.15.40:0/1574319571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c0b03a0 con 0x7f211c0cf040 2024-03-08T06:05:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.670+0000 7f2120e00640 1 -- 172.21.15.40:0/1574319571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c067cb0 con 0x7f211c0cf040 2024-03-08T06:05:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.671+0000 7f2112000640 1 -- 172.21.15.40:0/1574319571 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2114046020 con 0x7f211c0cf040 2024-03-08T06:05:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.672+0000 7f2112000640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f20fc06b500 0x7f20fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.672+0000 7f2112000640 1 -- 172.21.15.40:0/1574319571 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21140acab0 con 0x7f211c0cf040 2024-03-08T06:05:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.672+0000 7f2119e00640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f20fc06b500 0x7f20fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.673+0000 7f2119e00640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f20fc06b500 0x7f20fc06d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f210c006fd0 tx=0x7f210c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.676+0000 7f2112000640 1 -- 172.21.15.40:0/1574319571 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f211404d030 con 0x7f211c0cf040 2024-03-08T06:05:20.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.920+0000 7f2120e00640 1 -- 172.21.15.40:0/1574319571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f211c1427d0 con 0x7f211c0cf040 2024-03-08T06:05:20.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.921+0000 7f2112000640 1 -- 172.21.15.40:0/1574319571 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f211407b890 con 0x7f211c0cf040 2024-03-08T06:05:20.921 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T06:05:20.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 -- 172.21.15.40:0/1574319571 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f20fc06b500 msgr2=0x7f20fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:20.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f20fc06b500 0x7f20fc06d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f210c006fd0 tx=0x7f210c008040 comp rx=0 tx=0).stop 2024-03-08T06:05:20.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 -- 172.21.15.40:0/1574319571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0cf040 msgr2=0x7f211c1e98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:20.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0cf040 0x7f211c1e98e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2114036040 tx=0x7f2114004a40 comp rx=0 tx=0).stop 2024-03-08T06:05:20.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 -- 172.21.15.40:0/1574319571 shutdown_connections 2024-03-08T06:05:20.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f20fc06b500 0x7f20fc06d9c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:20.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0cf040 0x7f211c1e98e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:20.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211c0b46c0 0x7f211c141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:20.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 --2- 172.21.15.40:0/1574319571 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f211c071f50 0x7f211c0b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:20.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 -- 172.21.15.40:0/1574319571 >> 172.21.15.40:0/1574319571 conn(0x7f211c06ae30 msgr2=0x7f211c06b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:20.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 -- 172.21.15.40:0/1574319571 shutdown_connections 2024-03-08T06:05:20.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:20.924+0000 7f2120e00640 1 -- 172.21.15.40:0/1574319571 wait complete. 2024-03-08T06:05:20.946 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-08T06:05:20.946 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T06:05:20.946 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-03-08T06:05:20.946 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-08T06:05:20.946 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-08T06:05:20.946 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-08T06:05:20.946 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-08T06:05:20.946 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T06:05:20.947 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-08T06:05:20.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:20.947 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-03-08T06:05:21.007 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T06:05:21.007 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T06:05:21.007 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T06:05:21.007 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:21.007 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-03-08T06:05:21.007 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:21.007 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:05:21.007 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-03-08T06:05:21.007 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42493sec preferred_lft 42493sec 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 6a:91:a5:ab:52:6e brd ff:ff:ff:ff:ff:ff 2024-03-08T06:05:21.008 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T06:05:21.009 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:21.009 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::d48d:10ff:fe90:b28/64 scope link 2024-03-08T06:05:21.009 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:21.009 INFO:teuthology.orchestra.run.smithi053.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:05:21.009 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 6a:91:a5:ab:52:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T06:05:21.009 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::6891:a5ff:feab:526e/64 scope link 2024-03-08T06:05:21.009 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:21.009 INFO:teuthology.orchestra.run.smithi053.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T06:05:21.009 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ca:e3:52:62:e2:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-08T06:05:21.009 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::c8e3:52ff:fe62:e275/64 scope link 2024-03-08T06:05:21.009 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-03-08T06:05:21.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:21.010 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T06:05:21.105 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-03-08T06:05:21.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:21.110 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-03-08T06:05:21.165 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-03-08T06:05:21.165 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T06:05:21.165 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-08T06:05:21.165 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-08T06:05:21.165 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T06:05:21.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:21.165 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2024-03-08T06:05:21.165 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-03-08T06:05:21.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-08T06:05:21.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-08T06:05:21.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-08T06:05:21.165 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-03-08T06:05:21.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:21.355 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-03-08T06:05:21.407 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-08T06:05:21.407 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-08T06:05:21.408 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-08T06:05:21.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:21.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-03-08T06:05:21.498 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:05:21.499 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-08T06:05:21.555 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-08T06:05:21.555 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T06:05:21.555 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-08T06:05:21.610 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-08T06:05:21.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:21.610 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-08T06:05:21.748 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-08T06:05:21.748 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-08T06:05:21.748 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-03-08T06:05:21.771 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-08T06:05:21.771 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-08T06:05:21.771 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-03-08T06:05:21.772 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:05:21.772 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:05:21.799 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-08T06:05:21.800 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-08T06:05:21.800 INFO:teuthology.orchestra.run.smithi053.stdout:invalid new device string format 2024-03-08T06:05:21.800 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.53:3300,172.21.15.53:3301" 2024-03-08T06:05:21.800 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-08T06:05:21.800 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790" 2024-03-08T06:05:21.800 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.53:6789,172.21.15.53:6790:/ 2024-03-08T06:05:21.800 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c4e20909-a427-4e44-a3fb-93b068b9d1f6" will pass to kernel 2024-03-08T06:05:21.804 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-08T06:05:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2024-03-08T06:05:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T06:05:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T06:05:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T06:05:21.808 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T06:05:21.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:21.809 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-08T06:05:21.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:21.895 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:05:21.956 INFO:teuthology.orchestra.run.smithi053.stdout:client4655 2024-03-08T06:05:21.956 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:05:21.956 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:05:27.961 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:05:27.970 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:05:32.013 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T06:05:32.013 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-03-08T06:05:32.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.246+0000 7f47b0e00640 1 Processor -- start 2024-03-08T06:05:32.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.246+0000 7f47b0e00640 1 -- start start 2024-03-08T06:05:32.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.248+0000 7f47b0e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47ac071df0 0x7f47ac166600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.248+0000 7f47b0e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac15c460 con 0x7f47ac15bfb0 2024-03-08T06:05:32.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.248+0000 7f47b0e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac15c740 con 0x7f47ac145500 2024-03-08T06:05:32.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.248+0000 7f47b0e00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ac15ca20 con 0x7f47ac071df0 2024-03-08T06:05:32.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.248+0000 7f47aaa00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f47ac15bfb0 0x7f47ac0716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38162/0 (socket says 172.21.15.40:38162) 2024-03-08T06:05:32.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.248+0000 7f47aaa00640 1 -- 172.21.15.40:0/2354359928 learned_addr learned my addr 172.21.15.40:0/2354359928 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:05:32.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.248+0000 7f47abe00640 1 --2- 172.21.15.40:0/2354359928 >> v2:172.21.15.40:3300/0 conn(0x7f47ac071df0 0x7f47ac166600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.249+0000 7f47aa000640 1 -- 172.21.15.40:0/2354359928 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1251938156 0 0) 0x7f47ac15c460 con 0x7f47ac15bfb0 2024-03-08T06:05:32.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.249+0000 7f47abe00640 1 -- 172.21.15.40:0/2354359928 >> v1:172.21.15.53:6789/0 conn(0x7f47ac145500 legacy=0x7f47ac06ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:32.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.249+0000 7f47abe00640 1 -- 172.21.15.40:0/2354359928 >> v1:172.21.15.40:6789/0 conn(0x7f47ac15bfb0 legacy=0x7f47ac0716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:32.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.249+0000 7f47abe00640 1 -- 172.21.15.40:0/2354359928 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac15cd00 con 0x7f47ac071df0 2024-03-08T06:05:32.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.250+0000 7f47abe00640 1 --2- 172.21.15.40:0/2354359928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac071df0 0x7f47ac166600 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009d90 tx=0x7f47a402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a69ea9ad5c3db3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:32.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.250+0000 7f47aa000640 1 -- 172.21.15.40:0/2354359928 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4039070 con 0x7f47ac071df0 2024-03-08T06:05:32.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.250+0000 7f47aa000640 1 -- 172.21.15.40:0/2354359928 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a402cbd0 con 0x7f47ac071df0 2024-03-08T06:05:32.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.250+0000 7f47aa000640 1 -- 172.21.15.40:0/2354359928 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4034440 con 0x7f47ac071df0 2024-03-08T06:05:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.251+0000 7f47b0e00640 1 -- 172.21.15.40:0/2354359928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac071df0 msgr2=0x7f47ac166600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.251+0000 7f47b0e00640 1 --2- 172.21.15.40:0/2354359928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac071df0 0x7f47ac166600 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009d90 tx=0x7f47a402be20 comp rx=0 tx=0).stop 2024-03-08T06:05:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.251+0000 7f47b0e00640 1 -- 172.21.15.40:0/2354359928 shutdown_connections 2024-03-08T06:05:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.251+0000 7f47b0e00640 1 --2- 172.21.15.40:0/2354359928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac071df0 0x7f47ac166600 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.251+0000 7f47b0e00640 1 -- 172.21.15.40:0/2354359928 >> 172.21.15.40:0/2354359928 conn(0x7f47ac0b1850 msgr2=0x7f47ac0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.251+0000 7f47b0e00640 1 -- 172.21.15.40:0/2354359928 shutdown_connections 2024-03-08T06:05:32.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.251+0000 7f47b0e00640 1 -- 172.21.15.40:0/2354359928 wait complete. 2024-03-08T06:05:32.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.253+0000 7f47b0e00640 1 Processor -- start 2024-03-08T06:05:32.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.253+0000 7f47b0e00640 1 -- start start 2024-03-08T06:05:32.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.254+0000 7f47b0e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47ac071df0 0x7f47ac139240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.254+0000 7f47b0e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac145500 0x7f47ac13af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.254+0000 7f47b0e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47ac15bfb0 0x7f47ac0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.254+0000 7f47b0e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac166b40 con 0x7f47ac145500 2024-03-08T06:05:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.254+0000 7f47b0e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0cd760 con 0x7f47ac15bfb0 2024-03-08T06:05:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.254+0000 7f47b0e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac166cc0 con 0x7f47ac071df0 2024-03-08T06:05:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.254+0000 7f47aaa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac145500 0x7f47ac13af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.254+0000 7f47aaa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac145500 0x7f47ac13af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35322/0 (socket says 172.21.15.40:35322) 2024-03-08T06:05:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.254+0000 7f47aaa00640 1 -- 172.21.15.40:0/1834024105 learned_addr learned my addr 172.21.15.40:0/1834024105 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.255+0000 7f47aaa00640 1 -- 172.21.15.40:0/1834024105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47ac071df0 msgr2=0x7f47ac139240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.255+0000 7f47abe00640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47ac15bfb0 0x7f47ac0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.255+0000 7f47ab400640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47ac071df0 0x7f47ac139240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.255+0000 7f47aaa00640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47ac071df0 0x7f47ac139240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.255+0000 7f47aaa00640 1 -- 172.21.15.40:0/1834024105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47ac15bfb0 msgr2=0x7f47ac0b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.255+0000 7f47aaa00640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47ac15bfb0 0x7f47ac0b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.255+0000 7f47aaa00640 1 -- 172.21.15.40:0/1834024105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0b2a70 con 0x7f47ac145500 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.255+0000 7f47ab400640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47ac071df0 0x7f47ac139240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.255+0000 7f47abe00640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47ac15bfb0 0x7f47ac0b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.255+0000 7f47aaa00640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac145500 0x7f47ac13af80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f479c00c500 tx=0x7f479c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.256+0000 7f47a3400640 1 -- 172.21.15.40:0/1834024105 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c018070 con 0x7f47ac145500 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.256+0000 7f47a3400640 1 -- 172.21.15.40:0/1834024105 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c007580 con 0x7f47ac145500 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.256+0000 7f47a3400640 1 -- 172.21.15.40:0/1834024105 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c004a20 con 0x7f47ac145500 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.256+0000 7f47b0e00640 1 -- 172.21.15.40:0/1834024105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac0b2d50 con 0x7f47ac145500 2024-03-08T06:05:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.256+0000 7f47b0e00640 1 -- 172.21.15.40:0/1834024105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac0b3310 con 0x7f47ac145500 2024-03-08T06:05:32.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.258+0000 7f47a3400640 1 -- 172.21.15.40:0/1834024105 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f479c004bc0 con 0x7f47ac145500 2024-03-08T06:05:32.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.259+0000 7f47a3400640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f478806b4d0 0x7f478806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.260+0000 7f47a3400640 1 -- 172.21.15.40:0/1834024105 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f479c089530 con 0x7f47ac145500 2024-03-08T06:05:32.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.260+0000 7f47ab400640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f478806b4d0 0x7f478806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.260+0000 7f47b5015740 1 Processor -- start 2024-03-08T06:05:32.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.260+0000 7f47b5015740 1 -- start start 2024-03-08T06:05:32.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.261+0000 7f47ab400640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f478806b4d0 0x7f478806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4794003eb0 tx=0x7f4794009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:32.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.261+0000 7f47b5015740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59aff20 0x5632f5982ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.261+0000 7f47b5015740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59b0300 0x5632f59b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.261+0000 7f47b5015740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f5983400 0x5632f59b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.261+0000 7f47b5015740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f58ecd60 con 0x5632f5983400 2024-03-08T06:05:32.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.261+0000 7f47b5015740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f5895cf0 con 0x5632f59aff20 2024-03-08T06:05:32.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.261+0000 7f47b5015740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632f58e8e00 con 0x5632f59b0300 2024-03-08T06:05:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.262+0000 7f47abe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f5983400 0x5632f59b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.262+0000 7f47abe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f5983400 0x5632f59b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35324/0 (socket says 172.21.15.40:35324) 2024-03-08T06:05:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.262+0000 7f47abe00640 1 -- 172.21.15.40:0/390244548 learned_addr learned my addr 172.21.15.40:0/390244548 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.262+0000 7f47ab400640 1 --2- 172.21.15.40:0/390244548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59aff20 0x5632f5982ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.262+0000 7f47aaa00640 1 --2- 172.21.15.40:0/390244548 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59b0300 0x5632f59b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.262+0000 7f47abe00640 1 -- 172.21.15.40:0/390244548 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59b0300 msgr2=0x5632f59b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.262+0000 7f47abe00640 1 --2- 172.21.15.40:0/390244548 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59b0300 0x5632f59b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.262+0000 7f47abe00640 1 -- 172.21.15.40:0/390244548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59aff20 msgr2=0x5632f5982ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.262+0000 7f47abe00640 1 --2- 172.21.15.40:0/390244548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59aff20 0x5632f5982ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.262+0000 7f47abe00640 1 -- 172.21.15.40:0/390244548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632f56013f0 con 0x5632f5983400 2024-03-08T06:05:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.262+0000 7f47abe00640 1 --2- 172.21.15.40:0/390244548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f5983400 0x5632f59b31f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f47a4037040 tx=0x7f47a402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f3ae1cc88b41c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:32.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.263+0000 7f479be00640 1 -- 172.21.15.40:0/390244548 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4039040 con 0x5632f5983400 2024-03-08T06:05:32.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.263+0000 7f479be00640 1 -- 172.21.15.40:0/390244548 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a403f440 con 0x5632f5983400 2024-03-08T06:05:32.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.263+0000 7f479be00640 1 -- 172.21.15.40:0/390244548 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a403f7f0 con 0x5632f5983400 2024-03-08T06:05:32.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.263+0000 7f47b5015740 1 -- 172.21.15.40:0/390244548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f5983400 msgr2=0x5632f59b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.263+0000 7f47b5015740 1 --2- 172.21.15.40:0/390244548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f5983400 0x5632f59b31f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f47a4037040 tx=0x7f47a402e000 comp rx=0 tx=0).stop 2024-03-08T06:05:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.263+0000 7f47b5015740 1 -- 172.21.15.40:0/390244548 shutdown_connections 2024-03-08T06:05:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.263+0000 7f47b5015740 1 --2- 172.21.15.40:0/390244548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f5983400 0x5632f59b31f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.263+0000 7f47b5015740 1 --2- 172.21.15.40:0/390244548 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59b0300 0x5632f59b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.263+0000 7f47b5015740 1 --2- 172.21.15.40:0/390244548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59aff20 0x5632f5982ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.263+0000 7f47b5015740 1 -- 172.21.15.40:0/390244548 >> 172.21.15.40:0/390244548 conn(0x5632f5985130 msgr2=0x5632f59cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.264+0000 7f47b5015740 1 -- 172.21.15.40:0/390244548 shutdown_connections 2024-03-08T06:05:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.264+0000 7f47b5015740 1 -- 172.21.15.40:0/390244548 wait complete. 2024-03-08T06:05:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.264+0000 7f47b5015740 1 -- start start 2024-03-08T06:05:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.264+0000 7f47b5015740 1 Processor -- start 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.265+0000 7f47b5015740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 0x5632f59a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.265+0000 7f47b5015740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b0300 0x5632f59ab310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.265+0000 7f47b5015740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f59ab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.265+0000 7f47b5015740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f59877d0 con 0x5632f59b1f10 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.265+0000 7f47b5015740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f588b220 con 0x5632f59b0300 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.265+0000 7f47b5015740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632f598a010 con 0x5632f59aff20 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.266+0000 7f47b5015740 10 client.0 _ll_register_callbacks cb 0x5632f5600be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.266+0000 7f47abe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f59ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.266+0000 7f47abe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f59ab850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35336/0 (socket says 172.21.15.40:35336) 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.266+0000 7f47abe00640 1 -- 172.21.15.40:0/4064677984 learned_addr learned my addr 172.21.15.40:0/4064677984 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.266+0000 7f47abe00640 1 -- 172.21.15.40:0/4064677984 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 msgr2=0x5632f59a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.266+0000 7f47abe00640 1 --2- 172.21.15.40:0/4064677984 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 0x5632f59a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.266+0000 7f47abe00640 1 -- 172.21.15.40:0/4064677984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b0300 msgr2=0x5632f59ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.266+0000 7f47abe00640 1 --2- 172.21.15.40:0/4064677984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b0300 0x5632f59ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.266+0000 7f47abe00640 1 -- 172.21.15.40:0/4064677984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632f56cb7d0 con 0x5632f59b1f10 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.266+0000 7f47abe00640 1 --2- 172.21.15.40:0/4064677984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f59ab850 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009eb0 tx=0x7f47a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b363ec92fcf460b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.267+0000 7f4798c00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.267+0000 7f4798c00640 1 -- 172.21.15.40:0/4064677984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4039040 con 0x5632f59b1f10 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.267+0000 7f4798c00640 1 -- 172.21.15.40:0/4064677984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47a403fdc0 con 0x5632f59b1f10 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.267+0000 7f47b5015740 1 -- 172.21.15.40:0/4064677984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47a403f440 con 0x5632f59b1f10 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.267+0000 7f4798c00640 1 -- 172.21.15.40:0/4064677984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47a4059050 con 0x5632f59b1f10 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.267+0000 7f47b5015740 10 client.5259 fetch_fsmap learned FSMap version 16 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.267+0000 7f47b5015740 1 -- 172.21.15.40:0/4064677984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5632f59b2e20 con 0x5632f59b1f10 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.267+0000 7f4798c00640 1 -- 172.21.15.40:0/4064677984 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a403f440 con 0x5632f59b1f10 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.268+0000 7f4798c00640 1 -- 172.21.15.40:0/4064677984 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f47a405c020 con 0x5632f59b1f10 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.268+0000 7f47b5015740 10 client.5259 fetch_fsmap finished waiting for FSMap version 16 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.268+0000 7f47b5015740 10 client.5259 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.268+0000 7f47b5015740 20 client.5259 populate_metadata read hostname 'smithi040' 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.268+0000 7f47b5015740 1 --2- 172.21.15.40:0/4064677984 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x5632f59e1f20 0x5632f59e2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.268+0000 7f47b5015740 4 client.5259 mds_command: new command op to 5049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T06:05:32.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.268+0000 7f47b5015740 1 -- 172.21.15.40:0/4064677984 --> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632f58fa2a0 con 0x5632f59e1f20 2024-03-08T06:05:32.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.268+0000 7f47ab400640 1 --2- 172.21.15.40:0/4064677984 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x5632f59e1f20 0x5632f59e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.269+0000 7f47ab400640 1 --2- 172.21.15.40:0/4064677984 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x5632f59e1f20 0x5632f59e2330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25af8e34ba84a2a1 server_cookie=d20efc60637dbd5a in_seq=0 out_seq=0 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.269+0000 7f4798c00640 10 client.5259 ms_handle_connect on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.269+0000 7f4798c00640 1 -- 172.21.15.40:0/4064677984 <== mds.0 v2:172.21.15.40:6834/1742457959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632f58fa2a0 con 0x5632f59e1f20 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.269+0000 7f4798c00640 10 client.5259 handle_command_reply: tid=0 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.269+0000 7f47b5015740 1 client.5259 shutdown 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.269+0000 7f47b5015740 1 -- 172.21.15.40:0/4064677984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 msgr2=0x5632f59ab850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.269+0000 7f47b5015740 1 --2- 172.21.15.40:0/4064677984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f59ab850 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009eb0 tx=0x7f47a4004ab0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.270+0000 7f47b5015740 1 -- 172.21.15.40:0/4064677984 shutdown_connections 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.270+0000 7f47b5015740 1 --2- 172.21.15.40:0/4064677984 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x5632f59e1f20 0x5632f59e2330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.270+0000 7f47b5015740 1 --2- 172.21.15.40:0/4064677984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f59ab850 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.270+0000 7f47b5015740 1 --2- 172.21.15.40:0/4064677984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b0300 0x5632f59ab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.270+0000 7f47b5015740 1 --2- 172.21.15.40:0/4064677984 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 0x5632f59a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.270+0000 7f4798c00640 0 client.5259 ms_handle_reset on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.270+0000 7f47b5015740 1 -- 172.21.15.40:0/4064677984 >> 172.21.15.40:0/4064677984 conn(0x5632f5983400 msgr2=0x5632f59d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.270+0000 7f47b5015740 1 -- 172.21.15.40:0/4064677984 shutdown_connections 2024-03-08T06:05:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.270+0000 7f47b5015740 1 -- 172.21.15.40:0/4064677984 wait complete. 2024-03-08T06:05:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.270+0000 7f47b5015740 20 client.5259 trim_cache size 0 max 16384 2024-03-08T06:05:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.283+0000 7f47b5015740 1 Processor -- start 2024-03-08T06:05:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.283+0000 7f47b5015740 1 -- start start 2024-03-08T06:05:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.284+0000 7f47b5015740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 0x5632f59b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.284+0000 7f47b5015740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b0300 0x5632f5a3bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.284+0000 7f47b5015740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f5a3c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.284+0000 7f47b5015740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac1ffd30 con 0x5632f59b1f10 2024-03-08T06:05:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.284+0000 7f47b5015740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0ad830 con 0x5632f59b0300 2024-03-08T06:05:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.284+0000 7f47b5015740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0acc30 con 0x5632f59aff20 2024-03-08T06:05:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.284+0000 7f47abe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f5a3c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.284+0000 7f47abe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f5a3c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35342/0 (socket says 172.21.15.40:35342) 2024-03-08T06:05:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.284+0000 7f47abe00640 1 -- 172.21.15.40:0/1104315127 learned_addr learned my addr 172.21.15.40:0/1104315127 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.285+0000 7f47ab400640 1 --2- 172.21.15.40:0/1104315127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 0x5632f59b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.285+0000 7f47aaa00640 1 --2- 172.21.15.40:0/1104315127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b0300 0x5632f5a3bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.285+0000 7f47abe00640 1 -- 172.21.15.40:0/1104315127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 msgr2=0x5632f59b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.285+0000 7f47abe00640 1 --2- 172.21.15.40:0/1104315127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 0x5632f59b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.285+0000 7f47abe00640 1 -- 172.21.15.40:0/1104315127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b0300 msgr2=0x5632f5a3bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.285+0000 7f47abe00640 1 --2- 172.21.15.40:0/1104315127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b0300 0x5632f5a3bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.285+0000 7f47abe00640 1 -- 172.21.15.40:0/1104315127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632f59824e0 con 0x5632f59b1f10 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.285+0000 7f47abe00640 1 --2- 172.21.15.40:0/1104315127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f5a3c270 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009e80 tx=0x7f47a4049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345ae305a4722917 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f479b400640 1 -- 172.21.15.40:0/1104315127 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4061070 con 0x5632f59b1f10 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f479b400640 1 -- 172.21.15.40:0/1104315127 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a403fb00 con 0x5632f59b1f10 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f479b400640 1 -- 172.21.15.40:0/1104315127 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4039040 con 0x5632f59b1f10 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f47b5015740 1 -- 172.21.15.40:0/1104315127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 msgr2=0x5632f5a3c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f47b5015740 1 --2- 172.21.15.40:0/1104315127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f5a3c270 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009e80 tx=0x7f47a4049000 comp rx=0 tx=0).stop 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f47b5015740 1 -- 172.21.15.40:0/1104315127 shutdown_connections 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f47b5015740 1 --2- 172.21.15.40:0/1104315127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b1f10 0x5632f5a3c270 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f47b5015740 1 --2- 172.21.15.40:0/1104315127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b0300 0x5632f5a3bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f47b5015740 1 --2- 172.21.15.40:0/1104315127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 0x5632f59b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f47b5015740 1 -- 172.21.15.40:0/1104315127 >> 172.21.15.40:0/1104315127 conn(0x5632f5983400 msgr2=0x5632f597e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f47b5015740 1 -- 172.21.15.40:0/1104315127 shutdown_connections 2024-03-08T06:05:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.286+0000 7f47b5015740 1 -- 172.21.15.40:0/1104315127 wait complete. 2024-03-08T06:05:32.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.287+0000 7f47b5015740 1 -- start start 2024-03-08T06:05:32.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.287+0000 7f47b5015740 1 Processor -- start 2024-03-08T06:05:32.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.288+0000 7f47b5015740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 0x5632f5a257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.288+0000 7f47b5015740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b0300 0x5632f59a7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.288+0000 7f47b5015740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b1f10 0x5632f59dfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.288+0000 7f47b5015740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f5986770 con 0x5632f59b0300 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47b5015740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0ad230 con 0x5632f59b1f10 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47b5015740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632f5a25d00 con 0x5632f59aff20 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47b5015740 10 client.0 _ll_register_callbacks cb 0x7f47a4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47aaa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b0300 0x5632f59a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47aaa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b0300 0x5632f59a7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35358/0 (socket says 172.21.15.40:35358) 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47aaa00640 1 -- 172.21.15.40:0/4104896892 learned_addr learned my addr 172.21.15.40:0/4104896892 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47ab400640 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 0x5632f5a257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47aaa00640 1 -- 172.21.15.40:0/4104896892 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 msgr2=0x5632f5a257c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47aaa00640 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 0x5632f5a257c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47aaa00640 1 -- 172.21.15.40:0/4104896892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b1f10 msgr2=0x5632f59dfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47abe00640 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b1f10 0x5632f59dfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47aaa00640 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b1f10 0x5632f59dfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.289+0000 7f47aaa00640 1 -- 172.21.15.40:0/4104896892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632f59ae4e0 con 0x5632f59b0300 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f47aaa00640 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b0300 0x5632f59a7280 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f479c000f10 tx=0x7f479c05e1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05f6b16ef64cdd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f4798c00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f4798c00640 1 -- 172.21.15.40:0/4104896892 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c0892c0 con 0x5632f59b0300 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f4798c00640 1 -- 172.21.15.40:0/4104896892 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f479c0570f0 con 0x5632f59b0300 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f47b5015740 1 -- 172.21.15.40:0/4104896892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47a403fb00 con 0x5632f59b0300 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f4798c00640 1 -- 172.21.15.40:0/4104896892 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f479c05bdf0 con 0x5632f59b0300 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f47b5015740 10 client.5265 fetch_fsmap learned FSMap version 16 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f47b5015740 1 -- 172.21.15.40:0/4104896892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5632f5a379f0 con 0x5632f59b0300 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f4798c00640 1 -- 172.21.15.40:0/4104896892 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a403fb00 con 0x5632f59b0300 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f4798c00640 1 -- 172.21.15.40:0/4104896892 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f479c0539f0 con 0x5632f59b0300 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f47b5015740 10 client.5265 fetch_fsmap finished waiting for FSMap version 16 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f47b5015740 10 client.5265 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f47b5015740 20 client.5265 populate_metadata read hostname 'smithi040' 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.290+0000 7f47b5015740 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x5632f59a7810 0x5632f59e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.291+0000 7f47b5015740 4 client.5265 mds_command: new command op to 5049 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-08T06:05:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.291+0000 7f47b5015740 1 -- 172.21.15.40:0/4104896892 --> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x5632f58f45f0 con 0x5632f59a7810 2024-03-08T06:05:32.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.291+0000 7f47ab400640 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x5632f59a7810 0x5632f59e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.291+0000 7f47ab400640 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x5632f59a7810 0x5632f59e01a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a41fdc6e3f04b756 server_cookie=7b8f4e688c8deab1 in_seq=0 out_seq=0 2024-03-08T06:05:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.291+0000 7f4798c00640 10 client.5265 ms_handle_connect on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.292+0000 7f4798c00640 1 -- 172.21.15.40:0/4104896892 <== mds.0 v2:172.21.15.40:6834/1742457959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26754 (crc 0 0 0) 0x5632f58f45f0 con 0x5632f59a7810 2024-03-08T06:05:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.292+0000 7f4798c00640 10 client.5265 handle_command_reply: tid=0 2024-03-08T06:05:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.292+0000 7f47b5015740 1 client.5265 shutdown 2024-03-08T06:05:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.292+0000 7f47b5015740 1 -- 172.21.15.40:0/4104896892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b0300 msgr2=0x5632f59a7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.292+0000 7f47b5015740 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b0300 0x5632f59a7280 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f479c000f10 tx=0x7f479c05e1f0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.292+0000 7f47b5015740 1 -- 172.21.15.40:0/4104896892 shutdown_connections 2024-03-08T06:05:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.292+0000 7f47b5015740 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x5632f59a7810 0x5632f59e01a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f47b5015740 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5632f59b1f10 0x5632f59dfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f47b5015740 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5632f59b0300 0x5632f59a7280 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f47b5015740 1 --2- 172.21.15.40:0/4104896892 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x5632f59aff20 0x5632f5a257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f4798c00640 0 client.5265 ms_handle_reset on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f47b5015740 1 -- 172.21.15.40:0/4104896892 >> 172.21.15.40:0/4104896892 conn(0x5632f5983400 msgr2=0x5632f59ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f47b5015740 1 -- 172.21.15.40:0/4104896892 shutdown_connections 2024-03-08T06:05:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f47b5015740 1 -- 172.21.15.40:0/4104896892 wait complete. 2024-03-08T06:05:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f47b5015740 20 client.5265 trim_cache size 0 max 16384 2024-03-08T06:05:32.295 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:05:32.295 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker": [ 2024-03-08T06:05:32.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: "id": "0" 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: "id": "1" 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:05:32.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: "id": "2" 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": [ 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 31, 2024-03-08T06:05:32.297 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 30, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 8651, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 34057, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 9, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.014017890, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.001672491, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.007592219, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000625410, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 29, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.308671387, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.010643840 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2, 2024-03-08T06:05:32.298 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000216, 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000108 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 448 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": [ 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 43, 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 41, 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 12650, 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 127560, 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 8, 2024-03-08T06:05:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 4, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.019580302, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.007907823, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.010590207, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000990700, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 40, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.050241507, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.001256037 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 26, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000019735, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000759 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 94288 2024-03-08T06:05:32.300 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": [ 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 85, 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 71, 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 121074, 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 25962, 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 5, 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 4, 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.016970333, 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.004106587, 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.022223093, 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.002072823, 2024-03-08T06:05:32.301 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 71, 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.119353780, 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.001681039 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4, 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000003003, 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000750 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 112976, 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 13488 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": [ 2024-03-08T06:05:32.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": [ 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 46, 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.002607216, 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000056678 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": [ 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2, 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000024145, 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000012072 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.304 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 25, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 25, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 25, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.006566381, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000262655 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 12, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2024-03-08T06:05:32.305 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 10, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 0, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 10, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 0, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 10, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 0, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 1, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 2, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 2, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 49, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 49, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2024-03-08T06:05:32.306 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2024-03-08T06:05:32.307 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 1 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.308 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": [ 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2024-03-08T06:05:32.309 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_client_metrics": [ 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:32.310 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: "id": "5049" 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: "num_clients": 2 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_client_metrics-cephfs": [ 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4649", 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": "0" 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_hits": 5, 2024-03-08T06:05:32.311 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_miss": 18, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_read_latency": 0.000000000, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_write_latency": 0.000000000, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_lease_hits": 0, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_lease_miss": 0, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "opened_files": 0, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "opened_inodes": 1, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "pinned_icaps": 1, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "total_inodes": 1, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "total_read_ops": 0, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "total_read_size": 0, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "total_write_ops": 0, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "total_write_size": 0 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.312 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4655", 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": "0" 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_hits": 23, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_miss": 0, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_read_latency": 0.000000000, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_write_latency": 0.000000000, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_lease_hits": 0, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_lease_miss": 0, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "opened_files": 0, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "opened_inodes": 1, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "pinned_icaps": 1, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "total_inodes": 1, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "total_read_ops": 0, 2024-03-08T06:05:32.313 INFO:teuthology.orchestra.run.smithi040.stdout: "total_read_size": 0, 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "total_write_ops": 0, 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "total_write_size": 0 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": [ 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "evlrg": 0, 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 6, 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 6, 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2024-03-08T06:05:32.314 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 2, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 2, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "segmjr": 2, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 4196747, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 0, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.009253114, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.003084371 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 0 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.315 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": [ 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 13, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 13, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 0, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 12, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 12, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 0, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 10, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 10, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 0, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 2, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 2, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2024-03-08T06:05:32.316 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 40320, 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 215392 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": [ 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 25, 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 25, 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 2, 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.317 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 24, 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.005900599, 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000245858 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.318 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.319 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2024-03-08T06:05:32.320 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.321 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.322 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.323 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1, 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000665782, 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000665782 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.324 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2024-03-08T06:05:32.325 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": [ 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 2, 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 2, 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 0, 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 2, 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 24, 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 12, 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 7, 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: "mdthresh_evicted": 0 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.327 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": [ 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-03-08T06:05:32.328 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-03-08T06:05:32.329 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 225495, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 87, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-03-08T06:05:32.330 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 5312, 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 75, 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 52240, 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 213, 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": [ 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.331 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 41, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 13563, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 37, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 41, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 41, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.873988792, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.021316799 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 41, 2024-03-08T06:05:32.332 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 65 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 41, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 0, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 41, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 0, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 11, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 0, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 3, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 7, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 6, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2024-03-08T06:05:32.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 22, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2024-03-08T06:05:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 34, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2024-03-08T06:05:32.335 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 1, 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 7, 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 7, 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 15, 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 0, 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 1 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": [ 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 1, 2024-03-08T06:05:32.336 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": [ 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": [ 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.337 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed_ops": 0, 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-03-08T06:05:32.338 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 153, 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 124777, 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 153, 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 153, 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 124777, 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.339 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": [ 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 41, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 13563, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 41, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 13563, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.340 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": [ 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-03-08T06:05:32.341 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 41, 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 41, 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 41, 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 41, 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": [ 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.342 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.343 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x55c59da686e0": [ 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 6, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 2443, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 6, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 3, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 2443, 2024-03-08T06:05:32.344 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-03-08T06:05:32.345 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:32.345 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:32.345 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:32.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:32.345 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-03-08T06:05:32.345 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:05:32.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f47b0e00640 1 -- 172.21.15.40:0/1834024105 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f478806b4d0 msgr2=0x7f478806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:32.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f47b0e00640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f478806b4d0 0x7f478806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4794003eb0 tx=0x7f4794009040 comp rx=0 tx=0).stop 2024-03-08T06:05:32.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f47b0e00640 1 -- 172.21.15.40:0/1834024105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac145500 msgr2=0x7f47ac13af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:32.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.293+0000 7f47b0e00640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac145500 0x7f47ac13af80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f479c00c500 tx=0x7f479c00c9c0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.294+0000 7f47b0e00640 1 -- 172.21.15.40:0/1834024105 shutdown_connections 2024-03-08T06:05:32.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.294+0000 7f47b0e00640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f478806b4d0 0x7f478806d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.294+0000 7f47b0e00640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47ac15bfb0 0x7f47ac0b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.294+0000 7f47b0e00640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac145500 0x7f47ac13af80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.294+0000 7f47b0e00640 1 --2- 172.21.15.40:0/1834024105 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f47ac071df0 0x7f47ac139240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:32.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.294+0000 7f47b0e00640 1 -- 172.21.15.40:0/1834024105 >> 172.21.15.40:0/1834024105 conn(0x7f47ac0b1850 msgr2=0x7f47ac13f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:32.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.294+0000 7f47b0e00640 1 -- 172.21.15.40:0/1834024105 shutdown_connections 2024-03-08T06:05:32.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:32.294+0000 7f47b0e00640 1 -- 172.21.15.40:0/1834024105 wait complete. 2024-03-08T06:05:32.347 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:05:32.348 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:05:32.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:32.405 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:05:32.463 INFO:teuthology.orchestra.run.smithi040.stdout:client4649 2024-03-08T06:05:32.464 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:05:32.464 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:05:32.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:05:32.523 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T06:05:32.583 INFO:teuthology.orchestra.run.smithi053.stdout:client4655 2024-03-08T06:05:32.584 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:05:32.584 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> import uuid 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> n = 1000 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = False 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> if 0 > 0: 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> create_hard_links = True 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> path = "/home/ubuntu/cephtest/mnt.0/test_dir/test_file" 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> dpath = os.path.dirname(path) 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> fnameprefix = os.path.basename(path) 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> os.makedirs(dpath, exist_ok=True) 2024-03-08T06:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> try: 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> for i in range(n): 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> with open(fpath, '"'"'w'"'"') as f: 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> f.write(f"{i}") 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> if True: 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> f.flush() 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f.fileno()) 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> os.unlink(fpath) 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> if create_hard_links: 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> for j in range(0): 2024-03-08T06:05:32.645 DEBUG:teuthology.orchestra.run.smithi040:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-03-08T06:05:32.646 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2024-03-08T06:05:32.646 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(dirfd) 2024-03-08T06:05:32.646 DEBUG:teuthology.orchestra.run.smithi040:> finally: 2024-03-08T06:05:32.646 DEBUG:teuthology.orchestra.run.smithi040:> os.close(dirfd) 2024-03-08T06:05:32.646 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-03-08T06:05:42.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T06:05:42.496 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T06:05:42.496 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-03-08T06:05:42.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.765+0000 7fdde7e00640 1 Processor -- start 2024-03-08T06:05:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.766+0000 7fdde7e00640 1 -- start start 2024-03-08T06:05:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.768+0000 7fdde7e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdde01455b0 0x7fdde015c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.768+0000 7fdde7e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdde0071950 0x7fdde0071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.768+0000 7fdde7e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde00cd5a0 con 0x7fdde01604f0 2024-03-08T06:05:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.768+0000 7fdde7e00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde00ccda0 con 0x7fdde0071950 2024-03-08T06:05:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.768+0000 7fdde7e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde00cdc20 con 0x7fdde01455b0 2024-03-08T06:05:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.768+0000 7fdde5e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdde0071950 0x7fdde0071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.768+0000 7fdde5e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdde0071950 0x7fdde0071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55316/0 (socket says 172.21.15.40:55316) 2024-03-08T06:05:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.768+0000 7fdde5e00640 1 -- 172.21.15.40:0/114294783 learned_addr learned my addr 172.21.15.40:0/114294783 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:42.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.769+0000 7fdde5e00640 1 -- 172.21.15.40:0/114294783 >> v2:172.21.15.53:3300/0 conn(0x7fdde01455b0 msgr2=0x7fdde015c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:42.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.769+0000 7fdde5400640 1 --2- 172.21.15.40:0/114294783 >> v2:172.21.15.53:3300/0 conn(0x7fdde01455b0 0x7fdde015c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.769+0000 7fdde5e00640 1 --2- 172.21.15.40:0/114294783 >> v2:172.21.15.53:3300/0 conn(0x7fdde01455b0 0x7fdde015c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.769+0000 7fdde5e00640 1 -- 172.21.15.40:0/114294783 >> v1:172.21.15.53:6789/0 conn(0x7fdde01604f0 legacy=0x7fdde00cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.769+0000 7fdde5e00640 1 -- 172.21.15.40:0/114294783 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00ccfe0 con 0x7fdde0071950 2024-03-08T06:05:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.769+0000 7fdde5400640 1 --2- 172.21.15.40:0/114294783 >> v2:172.21.15.53:3300/0 conn(0x7fdde01455b0 0x7fdde015c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.770+0000 7fdde5e00640 1 --2- 172.21.15.40:0/114294783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdde0071950 0x7fdde0071d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdde006fc20 tx=0x7fddd802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba447a9a628b4c6a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.770+0000 7fdddfe00640 1 -- 172.21.15.40:0/114294783 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd802cb10 con 0x7fdde0071950 2024-03-08T06:05:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.770+0000 7fdddfe00640 1 -- 172.21.15.40:0/114294783 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd802ccb0 con 0x7fdde0071950 2024-03-08T06:05:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.770+0000 7fdddfe00640 1 -- 172.21.15.40:0/114294783 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd802ce60 con 0x7fdde0071950 2024-03-08T06:05:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.771+0000 7fdde7e00640 1 -- 172.21.15.40:0/114294783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdde0071950 msgr2=0x7fdde0071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.771+0000 7fdde7e00640 1 --2- 172.21.15.40:0/114294783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdde0071950 0x7fdde0071d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdde006fc20 tx=0x7fddd802bf70 comp rx=0 tx=0).stop 2024-03-08T06:05:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.771+0000 7fdde7e00640 1 -- 172.21.15.40:0/114294783 shutdown_connections 2024-03-08T06:05:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.771+0000 7fdde7e00640 1 --2- 172.21.15.40:0/114294783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdde0071950 0x7fdde0071d90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.771+0000 7fdde7e00640 1 --2- 172.21.15.40:0/114294783 >> v2:172.21.15.53:3300/0 conn(0x7fdde01455b0 0x7fdde015c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.771+0000 7fdde7e00640 1 -- 172.21.15.40:0/114294783 >> 172.21.15.40:0/114294783 conn(0x7fdde006ae30 msgr2=0x7fdde006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.771+0000 7fdde7e00640 1 -- 172.21.15.40:0/114294783 shutdown_connections 2024-03-08T06:05:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.771+0000 7fdde7e00640 1 -- 172.21.15.40:0/114294783 wait complete. 2024-03-08T06:05:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.773+0000 7fdde7e00640 1 Processor -- start 2024-03-08T06:05:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.773+0000 7fdde7e00640 1 -- start start 2024-03-08T06:05:42.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.774+0000 7fdde7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdde0071950 0x7fdde00b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.774+0000 7fdde7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde01455b0 0x7fdde013b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.774+0000 7fdde7e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdde01604f0 0x7fdde01394c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.774+0000 7fdde7e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00cd8b0 con 0x7fdde0071950 2024-03-08T06:05:42.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.774+0000 7fdde7e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde00722d0 con 0x7fdde01455b0 2024-03-08T06:05:42.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.774+0000 7fdde7e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde015c710 con 0x7fdde01604f0 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.774+0000 7fdde5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdde0071950 0x7fdde00b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.775+0000 7fdde5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdde0071950 0x7fdde00b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55332/0 (socket says 172.21.15.40:55332) 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.775+0000 7fdde5400640 1 -- 172.21.15.40:0/3293120028 learned_addr learned my addr 172.21.15.40:0/3293120028 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.775+0000 7fdde5e00640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdde01604f0 0x7fdde01394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.775+0000 7fdde4a00640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde01455b0 0x7fdde013b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.775+0000 7fdde5400640 1 -- 172.21.15.40:0/3293120028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdde01604f0 msgr2=0x7fdde01394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.775+0000 7fdde5400640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdde01604f0 0x7fdde01394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.775+0000 7fdde5400640 1 -- 172.21.15.40:0/3293120028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde01455b0 msgr2=0x7fdde013b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.775+0000 7fdde5400640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde01455b0 0x7fdde013b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.775+0000 7fdde5400640 1 -- 172.21.15.40:0/3293120028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00b35c0 con 0x7fdde0071950 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.775+0000 7fdde5e00640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdde01604f0 0x7fdde01394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:42.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.775+0000 7fdde5400640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdde0071950 0x7fdde00b2d70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c7f0 tx=0x7fddd400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:42.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.776+0000 7fdddd600640 1 -- 172.21.15.40:0/3293120028 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4015070 con 0x7fdde0071950 2024-03-08T06:05:42.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.776+0000 7fdddd600640 1 -- 172.21.15.40:0/3293120028 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4007a30 con 0x7fdde0071950 2024-03-08T06:05:42.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.776+0000 7fdddd600640 1 -- 172.21.15.40:0/3293120028 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4007e00 con 0x7fdde0071950 2024-03-08T06:05:42.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.776+0000 7fdde7e00640 1 -- 172.21.15.40:0/3293120028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde00b38a0 con 0x7fdde0071950 2024-03-08T06:05:42.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.776+0000 7fdde7e00640 1 -- 172.21.15.40:0/3293120028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde00b3e60 con 0x7fdde0071950 2024-03-08T06:05:42.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.778+0000 7fdddd600640 1 -- 172.21.15.40:0/3293120028 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddd4018070 con 0x7fdde0071950 2024-03-08T06:05:42.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.779+0000 7fdddd600640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fddc006b4d0 0x7fddc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.780+0000 7fdddd600640 1 -- 172.21.15.40:0/3293120028 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddd408a250 con 0x7fdde0071950 2024-03-08T06:05:42.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.780+0000 7fdde4a00640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fddc006b4d0 0x7fddc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.780+0000 7fddeab96740 1 Processor -- start 2024-03-08T06:05:42.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.780+0000 7fddeab96740 1 -- start start 2024-03-08T06:05:42.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.781+0000 7fdde4a00640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fddc006b4d0 0x7fddc006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fddcc007fd0 tx=0x7fddcc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:42.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.781+0000 7fddeab96740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97fefd50 0x55ac97fc2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.781+0000 7fddeab96740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac97ff2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.781+0000 7fddeab96740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fc3230 0x55ac97ff3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.781+0000 7fddeab96740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac97f2cd60 con 0x55ac97fc3230 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.781+0000 7fddeab96740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac97ed5cf0 con 0x55ac97fefd50 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.781+0000 7fddeab96740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac97f28e00 con 0x55ac97ff0130 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.781+0000 7fdde5e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fc3230 0x55ac97ff3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.781+0000 7fdde5e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fc3230 0x55ac97ff3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55338/0 (socket says 172.21.15.40:55338) 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.781+0000 7fdde5e00640 1 -- 172.21.15.40:0/122568967 learned_addr learned my addr 172.21.15.40:0/122568967 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.782+0000 7fdde5400640 1 --2- 172.21.15.40:0/122568967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97fefd50 0x55ac97fc2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.782+0000 7fdde4a00640 1 --2- 172.21.15.40:0/122568967 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac97ff2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.782+0000 7fdde5e00640 1 -- 172.21.15.40:0/122568967 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 msgr2=0x55ac97ff2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.782+0000 7fdde5e00640 1 --2- 172.21.15.40:0/122568967 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac97ff2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.782+0000 7fdde5e00640 1 -- 172.21.15.40:0/122568967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97fefd50 msgr2=0x55ac97fc2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.782+0000 7fdde5e00640 1 --2- 172.21.15.40:0/122568967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97fefd50 0x55ac97fc2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.782+0000 7fdde5e00640 1 -- 172.21.15.40:0/122568967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac97c413f0 con 0x55ac97fc3230 2024-03-08T06:05:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.782+0000 7fdde5e00640 1 --2- 172.21.15.40:0/122568967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fc3230 0x55ac97ff3020 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009f40 tx=0x7fddd802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346ab93ac3a3751f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.782+0000 7fddd1c00640 1 -- 172.21.15.40:0/122568967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd802ce60 con 0x55ac97fc3230 2024-03-08T06:05:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.782+0000 7fddd1c00640 1 -- 172.21.15.40:0/122568967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd802cb10 con 0x55ac97fc3230 2024-03-08T06:05:42.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.782+0000 7fddd1c00640 1 -- 172.21.15.40:0/122568967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd802ccc0 con 0x55ac97fc3230 2024-03-08T06:05:42.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.783+0000 7fddeab96740 1 -- 172.21.15.40:0/122568967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fc3230 msgr2=0x55ac97ff3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:42.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.783+0000 7fddeab96740 1 --2- 172.21.15.40:0/122568967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fc3230 0x55ac97ff3020 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009f40 tx=0x7fddd802e000 comp rx=0 tx=0).stop 2024-03-08T06:05:42.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.783+0000 7fddeab96740 1 -- 172.21.15.40:0/122568967 shutdown_connections 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.783+0000 7fddeab96740 1 --2- 172.21.15.40:0/122568967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fc3230 0x55ac97ff3020 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.783+0000 7fddeab96740 1 --2- 172.21.15.40:0/122568967 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac97ff2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.783+0000 7fddeab96740 1 --2- 172.21.15.40:0/122568967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97fefd50 0x55ac97fc2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.783+0000 7fddeab96740 1 -- 172.21.15.40:0/122568967 >> 172.21.15.40:0/122568967 conn(0x55ac97fc4f60 msgr2=0x55ac9800dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.783+0000 7fddeab96740 1 -- 172.21.15.40:0/122568967 shutdown_connections 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.783+0000 7fddeab96740 1 -- 172.21.15.40:0/122568967 wait complete. 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.784+0000 7fddeab96740 1 -- start start 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.784+0000 7fddeab96740 1 Processor -- start 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.785+0000 7fddeab96740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.785+0000 7fddeab96740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff0130 0x55ac97fe2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.785+0000 7fddeab96740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff68f0 0x55ac97ff9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.785+0000 7fddeab96740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac97fc7600 con 0x55ac97fefd50 2024-03-08T06:05:42.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.785+0000 7fddeab96740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac97ecb220 con 0x55ac97ff0130 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.785+0000 7fddeab96740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac97fc9e40 con 0x55ac97ff68f0 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.785+0000 7fddeab96740 10 client.0 _ll_register_callbacks cb 0x55ac97c40be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55344/0 (socket says 172.21.15.40:55344) 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde5400640 1 -- 172.21.15.40:0/49651515 learned_addr learned my addr 172.21.15.40:0/49651515 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde4a00640 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff0130 0x55ac97fe2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde5400640 1 -- 172.21.15.40:0/49651515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff68f0 msgr2=0x55ac97ff9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde5e00640 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff68f0 0x55ac97ff9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde5400640 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff68f0 0x55ac97ff9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde5400640 1 -- 172.21.15.40:0/49651515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff0130 msgr2=0x55ac97fe2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde5400640 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff0130 0x55ac97fe2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde5400640 1 -- 172.21.15.40:0/49651515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac97d0b7d0 con 0x55ac97fefd50 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde4a00640 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff0130 0x55ac97fe2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fdde5400640 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff7410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fddd4007410 tx=0x7fddd4059170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ea2f2cabd80f54 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:42.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.786+0000 7fddc6000640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddc6000640 1 -- 172.21.15.40:0/49651515 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd401f070 con 0x55ac97fefd50 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddc6000640 1 -- 172.21.15.40:0/49651515 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddd4057f80 con 0x55ac97fefd50 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddeab96740 1 -- 172.21.15.40:0/49651515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddd802cb10 con 0x55ac97fefd50 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddc6000640 1 -- 172.21.15.40:0/49651515 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddd4061040 con 0x55ac97fefd50 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddeab96740 10 client.5277 fetch_fsmap learned FSMap version 16 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddeab96740 1 -- 172.21.15.40:0/49651515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55ac97ff8340 con 0x55ac97fefd50 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddc6000640 1 -- 172.21.15.40:0/49651515 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd802cb10 con 0x55ac97fefd50 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddc6000640 1 -- 172.21.15.40:0/49651515 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fddd4088170 con 0x55ac97fefd50 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddeab96740 10 client.5277 fetch_fsmap finished waiting for FSMap version 16 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddeab96740 10 client.5277 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddeab96740 20 client.5277 populate_metadata read hostname 'smithi040' 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddeab96740 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x55ac9801e090 0x55ac9801e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddeab96740 4 client.5277 mds_command: new command op to 5049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T06:05:42.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.787+0000 7fddeab96740 1 -- 172.21.15.40:0/49651515 --> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac97f3a2a0 con 0x55ac9801e090 2024-03-08T06:05:42.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.788+0000 7fdde4a00640 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x55ac9801e090 0x55ac9801e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.788+0000 7fdde4a00640 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x55ac9801e090 0x55ac9801e4b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc8f5334792b82a server_cookie=5103253d86f895f9 in_seq=0 out_seq=0 2024-03-08T06:05:42.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.788+0000 7fddc6000640 10 client.5277 ms_handle_connect on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddc6000640 1 -- 172.21.15.40:0/49651515 <== mds.0 v2:172.21.15.40:6834/1742457959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ac97f3a2a0 con 0x55ac9801e090 2024-03-08T06:05:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddc6000640 10 client.5277 handle_command_reply: tid=0 2024-03-08T06:05:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddeab96740 1 client.5277 shutdown 2024-03-08T06:05:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddeab96740 1 -- 172.21.15.40:0/49651515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 msgr2=0x55ac97ff7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddeab96740 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff7410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fddd4007410 tx=0x7fddd4059170 comp rx=0 tx=0).stop 2024-03-08T06:05:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddeab96740 1 -- 172.21.15.40:0/49651515 shutdown_connections 2024-03-08T06:05:42.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddeab96740 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x55ac9801e090 0x55ac9801e4b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddeab96740 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff68f0 0x55ac97ff9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddeab96740 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff0130 0x55ac97fe2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddeab96740 1 --2- 172.21.15.40:0/49651515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff7410 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddc6000640 0 client.5277 ms_handle_reset on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:42.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.789+0000 7fddeab96740 1 -- 172.21.15.40:0/49651515 >> 172.21.15.40:0/49651515 conn(0x55ac97fc3230 msgr2=0x55ac980118f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:42.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.790+0000 7fddeab96740 1 -- 172.21.15.40:0/49651515 shutdown_connections 2024-03-08T06:05:42.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.790+0000 7fddeab96740 1 -- 172.21.15.40:0/49651515 wait complete. 2024-03-08T06:05:42.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.790+0000 7fddeab96740 20 client.5277 trim_cache size 0 max 16384 2024-03-08T06:05:42.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.803+0000 7fddeab96740 1 Processor -- start 2024-03-08T06:05:42.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.803+0000 7fddeab96740 1 -- start start 2024-03-08T06:05:42.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fddeab96740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac9806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fddeab96740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac9800d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fddeab96740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff68f0 0x55ac97fe2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fddeab96740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde0200060 con 0x55ac97fefd50 2024-03-08T06:05:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fddeab96740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde01f4e00 con 0x55ac97ff68f0 2024-03-08T06:05:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fddeab96740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde0142120 con 0x55ac97ff0130 2024-03-08T06:05:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fdde5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac9806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fdde5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac9806ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55360/0 (socket says 172.21.15.40:55360) 2024-03-08T06:05:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fdde5400640 1 -- 172.21.15.40:0/405012465 learned_addr learned my addr 172.21.15.40:0/405012465 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fdde5400640 1 -- 172.21.15.40:0/405012465 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 msgr2=0x55ac9800d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fdde5400640 1 --2- 172.21.15.40:0/405012465 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac9800d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fdde5400640 1 -- 172.21.15.40:0/405012465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff68f0 msgr2=0x55ac97fe2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fdde5400640 1 --2- 172.21.15.40:0/405012465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff68f0 0x55ac97fe2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.804+0000 7fdde5400640 1 -- 172.21.15.40:0/405012465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac97ff7780 con 0x55ac97fefd50 2024-03-08T06:05:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.805+0000 7fdde5400640 1 --2- 172.21.15.40:0/405012465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac9806ed70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fddd40588c0 tx=0x7fddd4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942d4af282742522 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.805+0000 7fddd1200640 1 -- 172.21.15.40:0/405012465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4077070 con 0x55ac97fefd50 2024-03-08T06:05:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.805+0000 7fddd1200640 1 -- 172.21.15.40:0/405012465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4077410 con 0x55ac97fefd50 2024-03-08T06:05:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.805+0000 7fddd1200640 1 -- 172.21.15.40:0/405012465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4057fc0 con 0x55ac97fefd50 2024-03-08T06:05:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.806+0000 7fddeab96740 1 -- 172.21.15.40:0/405012465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 msgr2=0x55ac9806ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.806+0000 7fddeab96740 1 --2- 172.21.15.40:0/405012465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac9806ed70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fddd40588c0 tx=0x7fddd4012000 comp rx=0 tx=0).stop 2024-03-08T06:05:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.806+0000 7fddeab96740 1 -- 172.21.15.40:0/405012465 shutdown_connections 2024-03-08T06:05:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.806+0000 7fddeab96740 1 --2- 172.21.15.40:0/405012465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff68f0 0x55ac97fe2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.806+0000 7fddeab96740 1 --2- 172.21.15.40:0/405012465 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac9800d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.806+0000 7fddeab96740 1 --2- 172.21.15.40:0/405012465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac9806ed70 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.806+0000 7fddeab96740 1 -- 172.21.15.40:0/405012465 >> 172.21.15.40:0/405012465 conn(0x55ac97fc3230 msgr2=0x55ac97fc1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:42.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.806+0000 7fddeab96740 1 -- 172.21.15.40:0/405012465 shutdown_connections 2024-03-08T06:05:42.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.806+0000 7fddeab96740 1 -- 172.21.15.40:0/405012465 wait complete. 2024-03-08T06:05:42.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.807+0000 7fddeab96740 1 -- start start 2024-03-08T06:05:42.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.807+0000 7fddeab96740 1 Processor -- start 2024-03-08T06:05:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fddeab96740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fddeab96740 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac97ff9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fddeab96740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff68f0 0x55ac98078370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fddeab96740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9801e470 con 0x55ac97fefd50 2024-03-08T06:05:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fddeab96740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde01f4800 con 0x55ac97ff68f0 2024-03-08T06:05:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fddeab96740 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac97fef4f0 con 0x55ac97ff0130 2024-03-08T06:05:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fddeab96740 10 client.0 _ll_register_callbacks cb 0x7fddd8009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T06:05:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fdde5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fdde5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55366/0 (socket says 172.21.15.40:55366) 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fdde5400640 1 -- 172.21.15.40:0/2907100804 learned_addr learned my addr 172.21.15.40:0/2907100804 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fdde5e00640 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff68f0 0x55ac98078370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.808+0000 7fdde4a00640 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac97ff9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fdde5400640 1 -- 172.21.15.40:0/2907100804 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 msgr2=0x55ac97ff9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fdde5400640 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac97ff9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fdde5400640 1 -- 172.21.15.40:0/2907100804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff68f0 msgr2=0x55ac98078370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fdde5400640 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff68f0 0x55ac98078370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fdde5400640 1 -- 172.21.15.40:0/2907100804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac9806c870 con 0x55ac97fefd50 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fdde5e00640 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff68f0 0x55ac98078370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fdde4a00640 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac97ff9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fdde5400640 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff8e10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fddd4058630 tx=0x7fddd40885b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6de3f5516cf48b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fddc6000640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fddc6000640 1 -- 172.21.15.40:0/2907100804 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4077040 con 0x55ac97fefd50 2024-03-08T06:05:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fddc6000640 1 -- 172.21.15.40:0/2907100804 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddd4071d70 con 0x55ac97fefd50 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fddeab96740 1 -- 172.21.15.40:0/2907100804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddd4077410 con 0x55ac97fefd50 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fddc6000640 1 -- 172.21.15.40:0/2907100804 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddd407c050 con 0x55ac97fefd50 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.809+0000 7fddeab96740 10 client.5283 fetch_fsmap learned FSMap version 16 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.810+0000 7fddeab96740 1 -- 172.21.15.40:0/2907100804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55ac9807c710 con 0x55ac97fefd50 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.810+0000 7fddc6000640 1 -- 172.21.15.40:0/2907100804 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4077410 con 0x55ac97fefd50 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.810+0000 7fddc6000640 1 -- 172.21.15.40:0/2907100804 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fddd407f020 con 0x55ac97fefd50 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.810+0000 7fddeab96740 10 client.5283 fetch_fsmap finished waiting for FSMap version 16 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.810+0000 7fddeab96740 10 client.5283 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.810+0000 7fddeab96740 20 client.5283 populate_metadata read hostname 'smithi040' 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.810+0000 7fddeab96740 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x55ac97fc8900 0x55ac980788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.810+0000 7fddeab96740 4 client.5283 mds_command: new command op to 5049 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.810+0000 7fddeab96740 1 -- 172.21.15.40:0/2907100804 --> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x55ac97f345f0 con 0x55ac97fc8900 2024-03-08T06:05:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.810+0000 7fdde4a00640 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x55ac97fc8900 0x55ac980788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:05:42.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.811+0000 7fdde4a00640 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x55ac97fc8900 0x55ac980788b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccebb5d1da306764 server_cookie=b64c58a2c36fd04d in_seq=0 out_seq=0 2024-03-08T06:05:42.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.811+0000 7fddc6000640 10 client.5283 ms_handle_connect on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.811+0000 7fddc6000640 1 -- 172.21.15.40:0/2907100804 <== mds.0 v2:172.21.15.40:6834/1742457959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26928 (crc 0 0 0) 0x55ac97f345f0 con 0x55ac97fc8900 2024-03-08T06:05:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.811+0000 7fddc6000640 10 client.5283 handle_command_reply: tid=0 2024-03-08T06:05:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 1 client.5283 shutdown 2024-03-08T06:05:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 1 -- 172.21.15.40:0/2907100804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 msgr2=0x55ac97ff8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff8e10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fddd4058630 tx=0x7fddd40885b0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 1 -- 172.21.15.40:0/2907100804 shutdown_connections 2024-03-08T06:05:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.40:6834/1742457959,v1:172.21.15.40:6836/1742457959] conn(0x55ac97fc8900 0x55ac980788b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac97ff68f0 0x55ac98078370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x55ac97ff0130 0x55ac97ff9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 1 --2- 172.21.15.40:0/2907100804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ac97fefd50 0x55ac97ff8e10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 1 -- 172.21.15.40:0/2907100804 >> 172.21.15.40:0/2907100804 conn(0x55ac97fc3230 msgr2=0x55ac97fe2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:42.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddc6000640 0 client.5283 ms_handle_reset on v2:172.21.15.40:6834/1742457959 2024-03-08T06:05:42.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 1 -- 172.21.15.40:0/2907100804 shutdown_connections 2024-03-08T06:05:42.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 1 -- 172.21.15.40:0/2907100804 wait complete. 2024-03-08T06:05:42.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.812+0000 7fddeab96740 20 client.5283 trim_cache size 0 max 16384 2024-03-08T06:05:42.845 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:05:42.845 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker": [ 2024-03-08T06:05:42.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: "id": "0" 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: "id": "1" 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.846 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: "id": "2" 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.847 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": [ 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1047, 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1046, 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 258847, 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 4659704, 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 11, 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 5, 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.272002627, 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.130154985, 2024-03-08T06:05:42.848 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.056539084, 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.059602667, 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1045, 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.387848019, 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000371146 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4, 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000469, 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000117 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 448 2024-03-08T06:05:42.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": [ 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 3058, 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 4048, 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 814070, 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 1861570, 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 8, 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 4, 2024-03-08T06:05:42.850 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 1.048740125, 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.756255902, 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.338642748, 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.002161893, 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4047, 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.352677944, 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000087145 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3981, 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.002264017, 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000568 2024-03-08T06:05:42.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 97872 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": [ 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 105, 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 81, 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 127765, 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 43697, 2024-03-08T06:05:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 5, 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 4, 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.021569852, 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.005795046, 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.023661241, 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.003441979, 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 81, 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.120158939, 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.001483443 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 8, 2024-03-08T06:05:42.853 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000006629, 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000828 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 113520, 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 14736 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": [ 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-03-08T06:05:42.854 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": [ 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2065, 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 1.312787700, 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000635732 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.855 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": [ 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2, 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000024145, 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000012072 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.856 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 1031, 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 1031, 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1031, 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 1.626685026, 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.001577774 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 12, 2024-03-08T06:05:42.857 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 1011, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 0, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 1011, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 0, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 1011, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 0, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 1002, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 1003, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 2, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 1061, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 1060, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2024-03-08T06:05:42.858 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 936, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 2695, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2024-03-08T06:05:42.859 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 1, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 1000, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 1001, 2024-03-08T06:05:42.860 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 1000, 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 3 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": [ 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2024-03-08T06:05:42.861 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2024-03-08T06:05:42.862 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_client_metrics": [ 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: "id": "5049" 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: "num_clients": 2 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.863 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_client_metrics-cephfs": [ 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4649", 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": "0" 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_hits": 9452, 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_miss": 19, 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_read_latency": 0.000000000, 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_write_latency": 0.000000000, 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_lease_hits": 943, 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_lease_miss": 0, 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "opened_files": 2, 2024-03-08T06:05:42.864 INFO:teuthology.orchestra.run.smithi040.stdout: "opened_inodes": 944, 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "pinned_icaps": 944, 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "total_inodes": 944, 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "total_read_ops": 0, 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "total_read_size": 0, 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "total_write_ops": 941, 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "total_write_size": 2713 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": { 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4655", 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": "0" 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_hits": 41, 2024-03-08T06:05:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_miss": 0, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_read_latency": 0.000000000, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_write_latency": 0.000000000, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_lease_hits": 0, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_lease_miss": 0, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "opened_files": 0, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "opened_inodes": 1, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "pinned_icaps": 1, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "total_inodes": 1, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "total_read_ops": 0, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "total_read_size": 0, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "total_write_ops": 0, 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: "total_write_size": 0 2024-03-08T06:05:42.866 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": [ 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "evlrg": 0, 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 2011, 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 2011, 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 4, 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2024-03-08T06:05:42.867 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 4, 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "segmjr": 2, 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 8402425, 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 0, 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1009, 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 4.972120305, 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.004927770 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 0 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.868 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": [ 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1014, 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 1014, 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 0, 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 13, 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 13, 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 0, 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 1011, 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 1011, 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 0, 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 1003, 2024-03-08T06:05:42.869 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 1003, 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 52864, 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 215392 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": [ 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 1032, 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 1031, 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 1004, 2024-03-08T06:05:42.870 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1000, 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 1.611834668, 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.001611834 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 29, 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.007114540, 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000245328 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.871 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.873 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1, 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.007070036, 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.007070036 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.874 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.875 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.877 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1, 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000665782, 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000665782 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.878 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2024-03-08T06:05:42.880 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": [ 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 2, 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 2, 2024-03-08T06:05:42.881 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 0, 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 2, 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 970, 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 485, 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 17, 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: "mdthresh_evicted": 0 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": [ 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.882 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-03-08T06:05:42.883 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-03-08T06:05:42.884 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 256463, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 671, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 1936, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 22, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 5312, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 75, 2024-03-08T06:05:42.885 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 1979841, 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 22653, 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": [ 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.886 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 1059, 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 4284132, 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 37, 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 1059, 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1059, 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 5.841924835, 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.005516454 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1059, 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 1093 2024-03-08T06:05:42.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 1059, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 0, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 1059, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 0, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 11, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 0, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 1009, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 8, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 7, 2024-03-08T06:05:42.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 22, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2024-03-08T06:05:42.889 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2024-03-08T06:05:42.890 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 34, 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 1, 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 7, 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 7, 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 35, 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 0, 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 1 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": [ 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.891 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 1, 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": [ 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": [ 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed_ops": 0, 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-08T06:05:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 4204, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 866854, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 4204, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 4204, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 866854, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": [ 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-03-08T06:05:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 1059, 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 4284132, 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 1059, 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 4284132, 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": [ 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 1059, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 1059, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 1059, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 1059, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": [ 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-03-08T06:05:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x55c59da686e0": [ 2024-03-08T06:05:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "labels": {}, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "counters": { 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 2011, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 4208121, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 2011, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 1009, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 4208121, 2024-03-08T06:05:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-03-08T06:05:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-03-08T06:05:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-03-08T06:05:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-03-08T06:05:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-03-08T06:05:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-03-08T06:05:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:05:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 -- 172.21.15.40:0/3293120028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fddc006b4d0 msgr2=0x7fddc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fddc006b4d0 0x7fddc006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fddcc007fd0 tx=0x7fddcc009040 comp rx=0 tx=0).stop 2024-03-08T06:05:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 -- 172.21.15.40:0/3293120028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdde0071950 msgr2=0x7fdde00b2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:05:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdde0071950 0x7fdde00b2d70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c7f0 tx=0x7fddd400cbf0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 -- 172.21.15.40:0/3293120028 shutdown_connections 2024-03-08T06:05:42.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fddc006b4d0 0x7fddc006d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdde01604f0 0x7fdde01394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde01455b0 0x7fdde013b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 --2- 172.21.15.40:0/3293120028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdde0071950 0x7fdde00b2d70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:05:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 -- 172.21.15.40:0/3293120028 >> 172.21.15.40:0/3293120028 conn(0x7fdde006ae30 msgr2=0x7fdde00af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:05:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 -- 172.21.15.40:0/3293120028 shutdown_connections 2024-03-08T06:05:42.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:05:42.813+0000 7fdde7e00640 1 -- 172.21.15.40:0/3293120028 wait complete. 2024-03-08T06:05:58.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:05:58.117 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:06:11.927 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-08T06:06:11.928 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T06:06:12.047 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T06:06:12.169 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-08T06:06:12.169 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:06:12.169 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:06:12.228 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:06:12.228 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:06:12.280 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:06:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.496+0000 7f7e68a00640 1 Processor -- start 2024-03-08T06:06:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.496+0000 7f7e68a00640 1 -- start start 2024-03-08T06:06:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.498+0000 7f7e68a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7e6415bea0 0x7f7e6415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.498+0000 7f7e68a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e64071df0 0x7f7e6406d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.498+0000 7f7e68a00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6415ceb0 con 0x7f7e64145520 2024-03-08T06:06:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.498+0000 7f7e68a00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6415c330 con 0x7f7e6415bea0 2024-03-08T06:06:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.498+0000 7f7e68a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6415c610 con 0x7f7e64071df0 2024-03-08T06:06:12.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.498+0000 7f7e63400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7e64145520 0x7f7e640716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35790/0 (socket says 172.21.15.40:35790) 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.498+0000 7f7e63400640 1 -- 172.21.15.40:0/425319543 learned_addr learned my addr 172.21.15.40:0/425319543 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.499+0000 7f7e62a00640 1 --2- 172.21.15.40:0/425319543 >> v2:172.21.15.53:3300/0 conn(0x7f7e6415bea0 0x7f7e6415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.499+0000 7f7e63e00640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e64071df0 0x7f7e6406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.499+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388971843 0 0) 0x7f7e6415ceb0 con 0x7f7e64145520 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.499+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e48003520 con 0x7f7e64145520 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.499+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2363961866 0 0) 0x7f7e48003520 con 0x7f7e64145520 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.499+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e64071df0 msgr2=0x7f7e6406d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.499+0000 7f7e62000640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e64071df0 0x7f7e6406d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.499+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 >> v2:172.21.15.53:3300/0 conn(0x7f7e6415bea0 msgr2=0x7f7e6415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.499+0000 7f7e62000640 1 --2- 172.21.15.40:0/425319543 >> v2:172.21.15.53:3300/0 conn(0x7f7e6415bea0 0x7f7e6415d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.499+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6415c8f0 con 0x7f7e64145520 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.499+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f7e54001670 con 0x7f7e64145520 2024-03-08T06:06:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 >> v1:172.21.15.40:6789/0 conn(0x7f7e64145520 legacy=0x7f7e640716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e62000640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e480039b0 0x7f7e48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e62000640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e64071df0 0x7f7e48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e62000640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e48004d40 0x7f7e480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6415c330 con 0x7f7e48004d40 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6415c610 con 0x7f7e480039b0 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4800b9c0 con 0x7f7e64071df0 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e63e00640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e48004d40 0x7f7e480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e63400640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e480039b0 0x7f7e48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e62a00640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e64071df0 0x7f7e48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e63e00640 1 -- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e64071df0 msgr2=0x7f7e48004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e63e00640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e64071df0 0x7f7e48004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e63e00640 1 -- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e480039b0 msgr2=0x7f7e48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e63e00640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e480039b0 0x7f7e48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.500+0000 7f7e63e00640 1 -- 172.21.15.40:0/425319543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e48003520 con 0x7f7e48004d40 2024-03-08T06:06:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.501+0000 7f7e63400640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e480039b0 0x7f7e48003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:12.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.502+0000 7f7e63e00640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e48004d40 0x7f7e480051c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009fc0 tx=0x7f7e48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daccfef825be4e42 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:12.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.502+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c039070 con 0x7f7e48004d40 2024-03-08T06:06:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.502+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c02cbd0 con 0x7f7e48004d40 2024-03-08T06:06:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.502+0000 7f7e62000640 1 -- 172.21.15.40:0/425319543 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c009c10 con 0x7f7e48004d40 2024-03-08T06:06:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.503+0000 7f7e68a00640 1 -- 172.21.15.40:0/425319543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e48004d40 msgr2=0x7f7e480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.503+0000 7f7e68a00640 1 --2- 172.21.15.40:0/425319543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e48004d40 0x7f7e480051c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009fc0 tx=0x7f7e48005830 comp rx=0 tx=0).stop 2024-03-08T06:06:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.503+0000 7f7e63e00640 1 -- 172.21.15.40:0/425319543 reap_dead start 2024-03-08T06:06:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.503+0000 7f7e68a00640 1 -- 172.21.15.40:0/425319543 shutdown_connections 2024-03-08T06:06:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.503+0000 7f7e68a00640 1 -- 172.21.15.40:0/425319543 >> 172.21.15.40:0/425319543 conn(0x7f7e640b1850 msgr2=0x7f7e640b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.503+0000 7f7e68a00640 1 -- 172.21.15.40:0/425319543 shutdown_connections 2024-03-08T06:06:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.503+0000 7f7e68a00640 1 -- 172.21.15.40:0/425319543 wait complete. 2024-03-08T06:06:12.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.505+0000 7f7e68a00640 1 Processor -- start 2024-03-08T06:06:12.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.505+0000 7f7e68a00640 1 -- start start 2024-03-08T06:06:12.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.506+0000 7f7e68a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e48004d40 0x7f7e64140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.506+0000 7f7e68a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e640adef0 0x7f7e640c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.506+0000 7f7e68a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ae370 0x7f7e640cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.506+0000 7f7e68a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e640af1c0 con 0x7f7e640ae370 2024-03-08T06:06:12.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.506+0000 7f7e68a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e640af040 con 0x7f7e48004d40 2024-03-08T06:06:12.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.506+0000 7f7e68a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6406e410 con 0x7f7e640adef0 2024-03-08T06:06:12.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.506+0000 7f7e63e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ae370 0x7f7e640cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.506+0000 7f7e63e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ae370 0x7f7e640cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54012/0 (socket says 172.21.15.40:54012) 2024-03-08T06:06:12.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.506+0000 7f7e63e00640 1 -- 172.21.15.40:0/1221389849 learned_addr learned my addr 172.21.15.40:0/1221389849 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e62a00640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e640adef0 0x7f7e640c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e63400640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e48004d40 0x7f7e64140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e63e00640 1 -- 172.21.15.40:0/1221389849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e640adef0 msgr2=0x7f7e640c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e63e00640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e640adef0 0x7f7e640c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e63e00640 1 -- 172.21.15.40:0/1221389849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e48004d40 msgr2=0x7f7e64140010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e63e00640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e48004d40 0x7f7e64140010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e63e00640 1 -- 172.21.15.40:0/1221389849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6413f250 con 0x7f7e640ae370 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e63e00640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ae370 0x7f7e640cbc10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c037040 tx=0x7f7e5c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e5b400640 1 -- 172.21.15.40:0/1221389849 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c039030 con 0x7f7e640ae370 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e5b400640 1 -- 172.21.15.40:0/1221389849 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c03a040 con 0x7f7e640ae370 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e5b400640 1 -- 172.21.15.40:0/1221389849 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c009c10 con 0x7f7e640ae370 2024-03-08T06:06:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e68a00640 1 -- 172.21.15.40:0/1221389849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6413f530 con 0x7f7e640ae370 2024-03-08T06:06:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.507+0000 7f7e68a00640 1 -- 172.21.15.40:0/1221389849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e640c61e0 con 0x7f7e640ae370 2024-03-08T06:06:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.509+0000 7f7e68a00640 1 -- 172.21.15.40:0/1221389849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e64075100 con 0x7f7e640ae370 2024-03-08T06:06:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.510+0000 7f7e5b400640 1 -- 172.21.15.40:0/1221389849 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e5c005d10 con 0x7f7e640ae370 2024-03-08T06:06:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.510+0000 7f7e5b400640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7e4406b4d0 0x7f7e4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.511+0000 7f7e5b400640 1 -- 172.21.15.40:0/1221389849 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e5c0abd90 con 0x7f7e640ae370 2024-03-08T06:06:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.511+0000 7f7e63400640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7e4406b4d0 0x7f7e4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.512+0000 7f7e63400640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7e4406b4d0 0x7f7e4406d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7e64143310 tx=0x7f7e54009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.516+0000 7f7e5b400640 1 -- 172.21.15.40:0/1221389849 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e5c07ac00 con 0x7f7e640ae370 2024-03-08T06:06:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.715+0000 7f7e68a00640 1 -- 172.21.15.40:0/1221389849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e640ce590 con 0x7f7e640ae370 2024-03-08T06:06:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.718+0000 7f7e5b400640 1 -- 172.21.15.40:0/1221389849 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16585 (secure 0 0 0) 0x7f7e5c07a500 con 0x7f7e640ae370 2024-03-08T06:06:12.720 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:12.720 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:05:12.978712+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:05:09.544904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T06:05:10.468274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:06:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 -- 172.21.15.40:0/1221389849 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7e4406b4d0 msgr2=0x7f7e4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7e4406b4d0 0x7f7e4406d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7e64143310 tx=0x7f7e54009040 comp rx=0 tx=0).stop 2024-03-08T06:06:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 -- 172.21.15.40:0/1221389849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ae370 msgr2=0x7f7e640cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ae370 0x7f7e640cbc10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c037040 tx=0x7f7e5c009750 comp rx=0 tx=0).stop 2024-03-08T06:06:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 -- 172.21.15.40:0/1221389849 shutdown_connections 2024-03-08T06:06:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f7e4406b4d0 0x7f7e4406d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ae370 0x7f7e640cbc10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e640adef0 0x7f7e640c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 --2- 172.21.15.40:0/1221389849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e48004d40 0x7f7e64140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 -- 172.21.15.40:0/1221389849 >> 172.21.15.40:0/1221389849 conn(0x7f7e640b1850 msgr2=0x7f7e6406ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 -- 172.21.15.40:0/1221389849 shutdown_connections 2024-03-08T06:06:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.721+0000 7f7e68a00640 1 -- 172.21.15.40:0/1221389849 wait complete. 2024-03-08T06:06:12.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:05:09.544904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:05:10.468274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:06:12.742 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:06:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.964+0000 7f148b200640 1 Processor -- start 2024-03-08T06:06:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.965+0000 7f148b200640 1 -- start start 2024-03-08T06:06:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.966+0000 7f148b200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f148c15f200 0x7f148c0d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.966+0000 7f148b200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f148c06be60 0x7f148c06cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.966+0000 7f148b200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f148c0cfb70 0x7f148c099c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.966+0000 7f148b200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148c15f5e0 con 0x7f148c0cfb70 2024-03-08T06:06:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.966+0000 7f148b200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148c0d0cf0 con 0x7f148c06be60 2024-03-08T06:06:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.966+0000 7f148b200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0d00e0 con 0x7f148c15f200 2024-03-08T06:06:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.966+0000 7f148a800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f148c0cfb70 0x7f148c099c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.967+0000 7f148a800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f148c0cfb70 0x7f148c099c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54014/0 (socket says 172.21.15.40:54014) 2024-03-08T06:06:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.967+0000 7f148a800640 1 -- 172.21.15.40:0/310795431 learned_addr learned my addr 172.21.15.40:0/310795431 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.967+0000 7f1489400640 1 --2- 172.21.15.40:0/310795431 >> v2:172.21.15.53:3300/0 conn(0x7f148c06be60 0x7f148c06cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.967+0000 7f1489e00640 1 --2- 172.21.15.40:0/310795431 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f148c15f200 0x7f148c0d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.967+0000 7f148a800640 1 -- 172.21.15.40:0/310795431 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f148c15f200 msgr2=0x7f148c0d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.967+0000 7f148a800640 1 --2- 172.21.15.40:0/310795431 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f148c15f200 0x7f148c0d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.967+0000 7f148a800640 1 -- 172.21.15.40:0/310795431 >> v2:172.21.15.53:3300/0 conn(0x7f148c06be60 msgr2=0x7f148c06cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.967+0000 7f148a800640 1 --2- 172.21.15.40:0/310795431 >> v2:172.21.15.53:3300/0 conn(0x7f148c06be60 0x7f148c06cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.967+0000 7f148a800640 1 -- 172.21.15.40:0/310795431 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c0d03c0 con 0x7f148c0cfb70 2024-03-08T06:06:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.969+0000 7f148a800640 1 --2- 172.21.15.40:0/310795431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c0cfb70 0x7f148c099c60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f148c072ae0 tx=0x7f148402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c5d2ff1429dac1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.969+0000 7f1488a00640 1 -- 172.21.15.40:0/310795431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148402ca70 con 0x7f148c0cfb70 2024-03-08T06:06:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.969+0000 7f1488a00640 1 -- 172.21.15.40:0/310795431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148402cc10 con 0x7f148c0cfb70 2024-03-08T06:06:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.969+0000 7f1488a00640 1 -- 172.21.15.40:0/310795431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14840343f0 con 0x7f148c0cfb70 2024-03-08T06:06:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.970+0000 7f148b200640 1 -- 172.21.15.40:0/310795431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c0cfb70 msgr2=0x7f148c099c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.970+0000 7f148b200640 1 --2- 172.21.15.40:0/310795431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c0cfb70 0x7f148c099c60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f148c072ae0 tx=0x7f148402bed0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.970+0000 7f148b200640 1 -- 172.21.15.40:0/310795431 shutdown_connections 2024-03-08T06:06:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.970+0000 7f148b200640 1 --2- 172.21.15.40:0/310795431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c0cfb70 0x7f148c099c60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.970+0000 7f148b200640 1 --2- 172.21.15.40:0/310795431 >> v2:172.21.15.53:3300/0 conn(0x7f148c06be60 0x7f148c06cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.970+0000 7f148b200640 1 --2- 172.21.15.40:0/310795431 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f148c15f200 0x7f148c0d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.970+0000 7f148b200640 1 -- 172.21.15.40:0/310795431 >> 172.21.15.40:0/310795431 conn(0x7f148c0b34f0 msgr2=0x7f148c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:12.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.970+0000 7f148b200640 1 -- 172.21.15.40:0/310795431 shutdown_connections 2024-03-08T06:06:12.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.970+0000 7f148b200640 1 -- 172.21.15.40:0/310795431 wait complete. 2024-03-08T06:06:12.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.972+0000 7f148b200640 1 Processor -- start 2024-03-08T06:06:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.972+0000 7f148b200640 1 -- start start 2024-03-08T06:06:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.973+0000 7f148b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c06be60 0x7f148c15f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.973+0000 7f148b200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f148c0cfb70 0x7f148c162530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.973+0000 7f148b200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c15f200 0x7f148c153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.973+0000 7f148b200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c09a1a0 con 0x7f148c15f200 2024-03-08T06:06:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.973+0000 7f148b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0d0980 con 0x7f148c06be60 2024-03-08T06:06:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.973+0000 7f148b200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148c09a320 con 0x7f148c0cfb70 2024-03-08T06:06:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.973+0000 7f148a800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c15f200 0x7f148c153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.973+0000 7f148a800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c15f200 0x7f148c153f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54030/0 (socket says 172.21.15.40:54030) 2024-03-08T06:06:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.973+0000 7f148a800640 1 -- 172.21.15.40:0/47016793 learned_addr learned my addr 172.21.15.40:0/47016793 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.973+0000 7f1489400640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f148c0cfb70 0x7f148c162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.973+0000 7f1489e00640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c06be60 0x7f148c15f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.974+0000 7f148a800640 1 -- 172.21.15.40:0/47016793 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f148c0cfb70 msgr2=0x7f148c162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.974+0000 7f148a800640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f148c0cfb70 0x7f148c162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.974+0000 7f148a800640 1 -- 172.21.15.40:0/47016793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c06be60 msgr2=0x7f148c15f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.974+0000 7f148a800640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c06be60 0x7f148c15f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.974+0000 7f148a800640 1 -- 172.21.15.40:0/47016793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c15e320 con 0x7f148c15f200 2024-03-08T06:06:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.974+0000 7f148a800640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c15f200 0x7f148c153f50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1484009f40 tx=0x7f1484004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.974+0000 7f1482000640 1 -- 172.21.15.40:0/47016793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484043020 con 0x7f148c15f200 2024-03-08T06:06:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.974+0000 7f1482000640 1 -- 172.21.15.40:0/47016793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148403e450 con 0x7f148c15f200 2024-03-08T06:06:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.974+0000 7f1482000640 1 -- 172.21.15.40:0/47016793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484009510 con 0x7f148c15f200 2024-03-08T06:06:12.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.974+0000 7f148b200640 1 -- 172.21.15.40:0/47016793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c15fd50 con 0x7f148c15f200 2024-03-08T06:06:12.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.975+0000 7f148b200640 1 -- 172.21.15.40:0/47016793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c160310 con 0x7f148c15f200 2024-03-08T06:06:12.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.976+0000 7f148b200640 1 -- 172.21.15.40:0/47016793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148c075090 con 0x7f148c15f200 2024-03-08T06:06:12.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.977+0000 7f1482000640 1 -- 172.21.15.40:0/47016793 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1484046020 con 0x7f148c15f200 2024-03-08T06:06:12.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.978+0000 7f1482000640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f146c06b500 0x7f146c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:12.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.978+0000 7f1482000640 1 -- 172.21.15.40:0/47016793 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14840ace20 con 0x7f148c15f200 2024-03-08T06:06:12.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.979+0000 7f1489e00640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f146c06b500 0x7f146c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:12.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.979+0000 7f1489e00640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f146c06b500 0x7f146c06d9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f147c006fd0 tx=0x7f147c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:12.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:12.984+0000 7f1482000640 1 -- 172.21.15.40:0/47016793 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f148404d020 con 0x7f148c15f200 2024-03-08T06:06:13.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.181+0000 7f148b200640 1 -- 172.21.15.40:0/47016793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f148c0b4350 con 0x7f148c15f200 2024-03-08T06:06:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.182+0000 7f1482000640 1 -- 172.21.15.40:0/47016793 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f148407bb70 con 0x7f148c15f200 2024-03-08T06:06:13.183 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:06:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 -- 172.21.15.40:0/47016793 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f146c06b500 msgr2=0x7f146c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f146c06b500 0x7f146c06d9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f147c006fd0 tx=0x7f147c008040 comp rx=0 tx=0).stop 2024-03-08T06:06:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 -- 172.21.15.40:0/47016793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c15f200 msgr2=0x7f148c153f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c15f200 0x7f148c153f50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1484009f40 tx=0x7f1484004a40 comp rx=0 tx=0).stop 2024-03-08T06:06:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 -- 172.21.15.40:0/47016793 shutdown_connections 2024-03-08T06:06:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f146c06b500 0x7f146c06d9c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f148c15f200 0x7f148c153f50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f148c0cfb70 0x7f148c162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 --2- 172.21.15.40:0/47016793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c06be60 0x7f148c15f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 -- 172.21.15.40:0/47016793 >> 172.21.15.40:0/47016793 conn(0x7f148c0b34f0 msgr2=0x7f148c0b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 -- 172.21.15.40:0/47016793 shutdown_connections 2024-03-08T06:06:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.185+0000 7f148b200640 1 -- 172.21.15.40:0/47016793 wait complete. 2024-03-08T06:06:13.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:06:13.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.435+0000 7f874ea00640 1 Processor -- start 2024-03-08T06:06:13.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.435+0000 7f874ea00640 1 -- start start 2024-03-08T06:06:13.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.437+0000 7f874ea00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8748071950 0x7f8748071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:13.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.437+0000 7f874ea00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87480cd5a0 con 0x7f87481604f0 2024-03-08T06:06:13.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.437+0000 7f874ea00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87480ccfe0 con 0x7f87481455b0 2024-03-08T06:06:13.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.437+0000 7f874ea00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87480ccda0 con 0x7f8748071950 2024-03-08T06:06:13.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.437+0000 7f874ca00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8748071950 0x7f8748071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:13.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.438+0000 7f874ca00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8748071950 0x7f8748071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54036/0 (socket says 172.21.15.40:54036) 2024-03-08T06:06:13.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.438+0000 7f873fe00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f87481604f0 0x7f87480cc690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35802/0 (socket says 172.21.15.40:35802) 2024-03-08T06:06:13.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.438+0000 7f874ca00640 1 -- 172.21.15.40:0/2216857310 learned_addr learned my addr 172.21.15.40:0/2216857310 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:13.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.438+0000 7f874ca00640 1 -- 172.21.15.40:0/2216857310 >> v1:172.21.15.53:6789/0 conn(0x7f87481455b0 legacy=0x7f874815c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:13.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.438+0000 7f874ca00640 1 -- 172.21.15.40:0/2216857310 >> v1:172.21.15.40:6789/0 conn(0x7f87481604f0 legacy=0x7f87480cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.438+0000 7f874ca00640 1 -- 172.21.15.40:0/2216857310 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87480cd2c0 con 0x7f8748071950 2024-03-08T06:06:13.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.438+0000 7f8747400640 1 -- 172.21.15.40:0/2216857310 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164711055 0 0) 0x7f87480cd5a0 con 0x7f87481604f0 2024-03-08T06:06:13.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.439+0000 7f874ca00640 1 --2- 172.21.15.40:0/2216857310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8748071950 0x7f8748071d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8740009e50 tx=0x7f874002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e331f218bb563fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:13.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.440+0000 7f8747400640 1 -- 172.21.15.40:0/2216857310 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874002cb20 con 0x7f8748071950 2024-03-08T06:06:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.440+0000 7f8747400640 1 -- 172.21.15.40:0/2216857310 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874002ccc0 con 0x7f8748071950 2024-03-08T06:06:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.440+0000 7f8747400640 1 -- 172.21.15.40:0/2216857310 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740034610 con 0x7f8748071950 2024-03-08T06:06:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.440+0000 7f874ea00640 1 -- 172.21.15.40:0/2216857310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8748071950 msgr2=0x7f8748071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.440+0000 7f874ea00640 1 --2- 172.21.15.40:0/2216857310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8748071950 0x7f8748071d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8740009e50 tx=0x7f874002bf80 comp rx=0 tx=0).stop 2024-03-08T06:06:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.440+0000 7f874ea00640 1 -- 172.21.15.40:0/2216857310 shutdown_connections 2024-03-08T06:06:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.440+0000 7f874ea00640 1 --2- 172.21.15.40:0/2216857310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8748071950 0x7f8748071d90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.440+0000 7f874ea00640 1 -- 172.21.15.40:0/2216857310 >> 172.21.15.40:0/2216857310 conn(0x7f874806ae30 msgr2=0x7f874806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.440+0000 7f874ea00640 1 -- 172.21.15.40:0/2216857310 shutdown_connections 2024-03-08T06:06:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.440+0000 7f874ea00640 1 -- 172.21.15.40:0/2216857310 wait complete. 2024-03-08T06:06:13.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.442+0000 7f874ea00640 1 Processor -- start 2024-03-08T06:06:13.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.442+0000 7f874ea00640 1 -- start start 2024-03-08T06:06:13.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.443+0000 7f874ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8748071950 0x7f874815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:13.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.443+0000 7f874ea00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87481455b0 0x7f874813ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:13.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.443+0000 7f874ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87481604f0 0x7f874813fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:13.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.443+0000 7f874ea00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87480722d0 con 0x7f8748071950 2024-03-08T06:06:13.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.443+0000 7f874ea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87480cdc20 con 0x7f87481604f0 2024-03-08T06:06:13.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.443+0000 7f874ea00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8748145990 con 0x7f87481455b0 2024-03-08T06:06:13.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.443+0000 7f8747e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8748071950 0x7f874815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:13.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.444+0000 7f8747e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8748071950 0x7f874815b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54052/0 (socket says 172.21.15.40:54052) 2024-03-08T06:06:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.444+0000 7f8747e00640 1 -- 172.21.15.40:0/451173970 learned_addr learned my addr 172.21.15.40:0/451173970 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.444+0000 7f874ca00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87481604f0 0x7f874813fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.444+0000 7f873fe00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87481455b0 0x7f874813ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.444+0000 7f8747e00640 1 -- 172.21.15.40:0/451173970 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87481455b0 msgr2=0x7f874813ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.444+0000 7f8747e00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87481455b0 0x7f874813ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.444+0000 7f8747e00640 1 -- 172.21.15.40:0/451173970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87481604f0 msgr2=0x7f874813fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.444+0000 7f8747e00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87481604f0 0x7f874813fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.444+0000 7f8747e00640 1 -- 172.21.15.40:0/451173970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87480b10c0 con 0x7f8748071950 2024-03-08T06:06:13.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.444+0000 7f8747e00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8748071950 0x7f874815b2b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f873800c430 tx=0x7f873800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.445+0000 7f8744c00640 1 -- 172.21.15.40:0/451173970 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738011790 con 0x7f8748071950 2024-03-08T06:06:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.445+0000 7f8744c00640 1 -- 172.21.15.40:0/451173970 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8738017070 con 0x7f8748071950 2024-03-08T06:06:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.445+0000 7f874ea00640 1 -- 172.21.15.40:0/451173970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87480b13a0 con 0x7f8748071950 2024-03-08T06:06:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.445+0000 7f8744c00640 1 -- 172.21.15.40:0/451173970 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738011b70 con 0x7f8748071950 2024-03-08T06:06:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.445+0000 7f874ea00640 1 -- 172.21.15.40:0/451173970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874815b7f0 con 0x7f8748071950 2024-03-08T06:06:13.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.446+0000 7f874ea00640 1 -- 172.21.15.40:0/451173970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8748067cb0 con 0x7f8748071950 2024-03-08T06:06:13.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.447+0000 7f8744c00640 1 -- 172.21.15.40:0/451173970 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8738011d10 con 0x7f8748071950 2024-03-08T06:06:13.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.448+0000 7f8744c00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f872406b4d0 0x7f872406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:13.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.448+0000 7f8744c00640 1 -- 172.21.15.40:0/451173970 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f873808a110 con 0x7f8748071950 2024-03-08T06:06:13.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.448+0000 7f873fe00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f872406b4d0 0x7f872406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:13.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.449+0000 7f873fe00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f872406b4d0 0x7f872406d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f87481431e0 tx=0x7f8730009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:13.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.453+0000 7f8744c00640 1 -- 172.21.15.40:0/451173970 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8738058ef0 con 0x7f8748071950 2024-03-08T06:06:13.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.654+0000 7f874ea00640 1 -- 172.21.15.40:0/451173970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f874815bf20 con 0x7f8748071950 2024-03-08T06:06:13.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.655+0000 7f8744c00640 1 -- 172.21.15.40:0/451173970 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87380587f0 con 0x7f8748071950 2024-03-08T06:06:13.656 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.657+0000 7f874ea00640 1 -- 172.21.15.40:0/451173970 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f872406b4d0 msgr2=0x7f872406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.657+0000 7f874ea00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f872406b4d0 0x7f872406d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f87481431e0 tx=0x7f8730009040 comp rx=0 tx=0).stop 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.657+0000 7f874ea00640 1 -- 172.21.15.40:0/451173970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8748071950 msgr2=0x7f874815b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.657+0000 7f874ea00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8748071950 0x7f874815b2b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f873800c430 tx=0x7f873800c8f0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.657+0000 7f874ea00640 1 -- 172.21.15.40:0/451173970 shutdown_connections 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.657+0000 7f874ea00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f872406b4d0 0x7f872406d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.657+0000 7f874ea00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87481604f0 0x7f874813fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.657+0000 7f874ea00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87481455b0 0x7f874813ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.657+0000 7f874ea00640 1 --2- 172.21.15.40:0/451173970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8748071950 0x7f874815b2b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.657+0000 7f874ea00640 1 -- 172.21.15.40:0/451173970 >> 172.21.15.40:0/451173970 conn(0x7f874806ae30 msgr2=0x7f874806b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.657+0000 7f874ea00640 1 -- 172.21.15.40:0/451173970 shutdown_connections 2024-03-08T06:06:13.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.658+0000 7f874ea00640 1 -- 172.21.15.40:0/451173970 wait complete. 2024-03-08T06:06:13.678 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:06:13.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.903+0000 7f1b5f200640 1 Processor -- start 2024-03-08T06:06:13.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.904+0000 7f1b5f200640 1 -- start start 2024-03-08T06:06:13.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.905+0000 7f1b5f200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b60147040 0x7f1b6015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:13.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.905+0000 7f1b5f200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6015e840 con 0x7f1b600716e0 2024-03-08T06:06:13.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.905+0000 7f1b5f200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6015eb20 con 0x7f1b600b2970 2024-03-08T06:06:13.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.905+0000 7f1b5f200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6015ee00 con 0x7f1b60147040 2024-03-08T06:06:13.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.906+0000 7f1b5de00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b60147040 0x7f1b6015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:13.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.906+0000 7f1b5de00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b60147040 0x7f1b6015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54060/0 (socket says 172.21.15.40:54060) 2024-03-08T06:06:13.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.906+0000 7f1b5de00640 1 -- 172.21.15.40:0/3838855788 learned_addr learned my addr 172.21.15.40:0/3838855788 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:13.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.906+0000 7f1b5de00640 1 -- 172.21.15.40:0/3838855788 >> v1:172.21.15.53:6789/0 conn(0x7f1b600b2970 legacy=0x7f1b6006e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:13.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.906+0000 7f1b5de00640 1 -- 172.21.15.40:0/3838855788 >> v1:172.21.15.40:6789/0 conn(0x7f1b600716e0 legacy=0x7f1b600ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.906+0000 7f1b5de00640 1 -- 172.21.15.40:0/3838855788 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b600ae0d0 con 0x7f1b60147040 2024-03-08T06:06:13.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.907+0000 7f1b5de00640 1 --2- 172.21.15.40:0/3838855788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b60147040 0x7f1b6015e1a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009e80 tx=0x7f1b5002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbdeb25743dfdb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:13.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.908+0000 7f1b5d400640 1 -- 172.21.15.40:0/3838855788 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5002cb00 con 0x7f1b60147040 2024-03-08T06:06:13.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.908+0000 7f1b5d400640 1 -- 172.21.15.40:0/3838855788 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5002cca0 con 0x7f1b60147040 2024-03-08T06:06:13.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.908+0000 7f1b5d400640 1 -- 172.21.15.40:0/3838855788 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50034610 con 0x7f1b60147040 2024-03-08T06:06:13.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.908+0000 7f1b5f200640 1 -- 172.21.15.40:0/3838855788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b60147040 msgr2=0x7f1b6015e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.908+0000 7f1b5f200640 1 --2- 172.21.15.40:0/3838855788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b60147040 0x7f1b6015e1a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009e80 tx=0x7f1b5002bf60 comp rx=0 tx=0).stop 2024-03-08T06:06:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.909+0000 7f1b5f200640 1 -- 172.21.15.40:0/3838855788 shutdown_connections 2024-03-08T06:06:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.909+0000 7f1b5f200640 1 --2- 172.21.15.40:0/3838855788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b60147040 0x7f1b6015e1a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.909+0000 7f1b5f200640 1 -- 172.21.15.40:0/3838855788 >> 172.21.15.40:0/3838855788 conn(0x7f1b600b1850 msgr2=0x7f1b600b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.909+0000 7f1b5f200640 1 -- 172.21.15.40:0/3838855788 shutdown_connections 2024-03-08T06:06:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.909+0000 7f1b5f200640 1 -- 172.21.15.40:0/3838855788 wait complete. 2024-03-08T06:06:13.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.910+0000 7f1b5f200640 1 Processor -- start 2024-03-08T06:06:13.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.911+0000 7f1b5f200640 1 -- start start 2024-03-08T06:06:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.911+0000 7f1b5f200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b600716e0 0x7f1b6015fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b600b2970 0x7f1b6015d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5f200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b60147040 0x7f1b601445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:13.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5f200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60071cd0 con 0x7f1b600716e0 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60071b50 con 0x7f1b600b2970 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5f200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60071e50 con 0x7f1b60147040 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b600716e0 0x7f1b6015fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b600716e0 0x7f1b6015fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54064/0 (socket says 172.21.15.40:54064) 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5de00640 1 -- 172.21.15.40:0/1806495829 learned_addr learned my addr 172.21.15.40:0/1806495829 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b55400640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b600b2970 0x7f1b6015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5e800640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b60147040 0x7f1b601445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5de00640 1 -- 172.21.15.40:0/1806495829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b60147040 msgr2=0x7f1b601445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5de00640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b60147040 0x7f1b601445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5de00640 1 -- 172.21.15.40:0/1806495829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b600b2970 msgr2=0x7f1b6015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5de00640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b600b2970 0x7f1b6015d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5de00640 1 -- 172.21.15.40:0/1806495829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b601410f0 con 0x7f1b600716e0 2024-03-08T06:06:13.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.912+0000 7f1b5e800640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b60147040 0x7f1b601445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.913+0000 7f1b5de00640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b600716e0 0x7f1b6015fb30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1b50000f80 tx=0x7f1b50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.913+0000 7f1b56a00640 1 -- 172.21.15.40:0/1806495829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50043020 con 0x7f1b600716e0 2024-03-08T06:06:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.913+0000 7f1b56a00640 1 -- 172.21.15.40:0/1806495829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5002ce50 con 0x7f1b600716e0 2024-03-08T06:06:13.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.913+0000 7f1b56a00640 1 -- 172.21.15.40:0/1806495829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5003e690 con 0x7f1b600716e0 2024-03-08T06:06:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.913+0000 7f1b5f200640 1 -- 172.21.15.40:0/1806495829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b601413d0 con 0x7f1b600716e0 2024-03-08T06:06:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.913+0000 7f1b5f200640 1 -- 172.21.15.40:0/1806495829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b60141990 con 0x7f1b600716e0 2024-03-08T06:06:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.914+0000 7f1b5f200640 1 -- 172.21.15.40:0/1806495829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b60075090 con 0x7f1b600716e0 2024-03-08T06:06:13.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.915+0000 7f1b56a00640 1 -- 172.21.15.40:0/1806495829 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b50046020 con 0x7f1b600716e0 2024-03-08T06:06:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.916+0000 7f1b56a00640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1b3c06b4d0 0x7f1b3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.916+0000 7f1b56a00640 1 -- 172.21.15.40:0/1806495829 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b500ac9e0 con 0x7f1b600716e0 2024-03-08T06:06:13.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.916+0000 7f1b55400640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1b3c06b4d0 0x7f1b3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.917+0000 7f1b55400640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1b3c06b4d0 0x7f1b3c06d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1b48006fd0 tx=0x7f1b48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:13.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:13.921+0000 7f1b56a00640 1 -- 172.21.15.40:0/1806495829 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b5004d030 con 0x7f1b600716e0 2024-03-08T06:06:14.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.118+0000 7f1b5f200640 1 -- 172.21.15.40:0/1806495829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b60161150 con 0x7f1b600716e0 2024-03-08T06:06:14.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.119+0000 7f1b56a00640 1 -- 172.21.15.40:0/1806495829 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b5007b7c0 con 0x7f1b600716e0 2024-03-08T06:06:14.120 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.121+0000 7f1b5f200640 1 -- 172.21.15.40:0/1806495829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1b3c06b4d0 msgr2=0x7f1b3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.121+0000 7f1b5f200640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1b3c06b4d0 0x7f1b3c06d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1b48006fd0 tx=0x7f1b48008040 comp rx=0 tx=0).stop 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.121+0000 7f1b5f200640 1 -- 172.21.15.40:0/1806495829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b600716e0 msgr2=0x7f1b6015fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.121+0000 7f1b5f200640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b600716e0 0x7f1b6015fb30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1b50000f80 tx=0x7f1b50004a40 comp rx=0 tx=0).stop 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.121+0000 7f1b5f200640 1 -- 172.21.15.40:0/1806495829 shutdown_connections 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.121+0000 7f1b5f200640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1b3c06b4d0 0x7f1b3c06d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.121+0000 7f1b5f200640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b60147040 0x7f1b601445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.121+0000 7f1b5f200640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b600b2970 0x7f1b6015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.121+0000 7f1b5f200640 1 --2- 172.21.15.40:0/1806495829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b600716e0 0x7f1b6015fb30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.121+0000 7f1b5f200640 1 -- 172.21.15.40:0/1806495829 >> 172.21.15.40:0/1806495829 conn(0x7f1b600b1850 msgr2=0x7f1b600ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.121+0000 7f1b5f200640 1 -- 172.21.15.40:0/1806495829 shutdown_connections 2024-03-08T06:06:14.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.122+0000 7f1b5f200640 1 -- 172.21.15.40:0/1806495829 wait complete. 2024-03-08T06:06:14.140 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:06:14.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.357+0000 7f1e0a600640 1 Processor -- start 2024-03-08T06:06:14.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.358+0000 7f1e0a600640 1 -- start start 2024-03-08T06:06:14.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.359+0000 7f1e0a600640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e040cd730 0x7f1e040cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:14.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.359+0000 7f1e0a600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1e040717d0 0x7f1e04071c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:14.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e0a600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e040cc690 con 0x7f1e04145250 2024-03-08T06:06:14.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e0a600640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e04072180 con 0x7f1e040cd730 2024-03-08T06:06:14.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e0a600640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e040cc970 con 0x7f1e040717d0 2024-03-08T06:06:14.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e02a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e040cd730 0x7f1e040cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:14.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e02a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e040cd730 0x7f1e040cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54074/0 (socket says 172.21.15.40:54074) 2024-03-08T06:06:14.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e02a00640 1 -- 172.21.15.40:0/801325260 learned_addr learned my addr 172.21.15.40:0/801325260 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:14.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e03e00640 1 --2- 172.21.15.40:0/801325260 >> v2:172.21.15.53:3300/0 conn(0x7f1e040717d0 0x7f1e04071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:14.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e02a00640 1 -- 172.21.15.40:0/801325260 >> v2:172.21.15.53:3300/0 conn(0x7f1e040717d0 msgr2=0x7f1e04071c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:14.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e02a00640 1 --2- 172.21.15.40:0/801325260 >> v2:172.21.15.53:3300/0 conn(0x7f1e040717d0 0x7f1e04071c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e02a00640 1 -- 172.21.15.40:0/801325260 >> v1:172.21.15.53:6789/0 conn(0x7f1e04145250 legacy=0x7f1e0406eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:14.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e02a00640 1 -- 172.21.15.40:0/801325260 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e040ccc50 con 0x7f1e040cd730 2024-03-08T06:06:14.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.360+0000 7f1e03e00640 1 --2- 172.21.15.40:0/801325260 >> v2:172.21.15.53:3300/0 conn(0x7f1e040717d0 0x7f1e04071c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:14.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.361+0000 7f1e02a00640 1 --2- 172.21.15.40:0/801325260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e040cd730 0x7f1e040cc0b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1e040cf3c0 tx=0x7f1dec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca61692158f7e844 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:14.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.362+0000 7f1e02000640 1 -- 172.21.15.40:0/801325260 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec035020 con 0x7f1e040cd730 2024-03-08T06:06:14.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.362+0000 7f1e02000640 1 -- 172.21.15.40:0/801325260 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec031870 con 0x7f1e040cd730 2024-03-08T06:06:14.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.362+0000 7f1e02000640 1 -- 172.21.15.40:0/801325260 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec031c40 con 0x7f1e040cd730 2024-03-08T06:06:14.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.362+0000 7f1e0a600640 1 -- 172.21.15.40:0/801325260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e040cd730 msgr2=0x7f1e040cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:14.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.362+0000 7f1e0a600640 1 --2- 172.21.15.40:0/801325260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e040cd730 0x7f1e040cc0b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1e040cf3c0 tx=0x7f1dec02b910 comp rx=0 tx=0).stop 2024-03-08T06:06:14.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.362+0000 7f1e0a600640 1 -- 172.21.15.40:0/801325260 shutdown_connections 2024-03-08T06:06:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.362+0000 7f1e0a600640 1 --2- 172.21.15.40:0/801325260 >> v2:172.21.15.53:3300/0 conn(0x7f1e040717d0 0x7f1e04071c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.362+0000 7f1e0a600640 1 --2- 172.21.15.40:0/801325260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e040cd730 0x7f1e040cc0b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.362+0000 7f1e0a600640 1 -- 172.21.15.40:0/801325260 >> 172.21.15.40:0/801325260 conn(0x7f1e0406ae30 msgr2=0x7f1e0406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.363+0000 7f1e0a600640 1 -- 172.21.15.40:0/801325260 shutdown_connections 2024-03-08T06:06:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.363+0000 7f1e0a600640 1 -- 172.21.15.40:0/801325260 wait complete. 2024-03-08T06:06:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.364+0000 7f1e0a600640 1 Processor -- start 2024-03-08T06:06:14.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.364+0000 7f1e0a600640 1 -- start start 2024-03-08T06:06:14.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.365+0000 7f1e0a600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e040717d0 0x7f1e0415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:14.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.365+0000 7f1e0a600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e04145250 0x7f1e0413cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:14.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.365+0000 7f1e0a600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04153280 0x7f1e0413e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:14.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.365+0000 7f1e0a600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e04145630 con 0x7f1e04153280 2024-03-08T06:06:14.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.365+0000 7f1e0a600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040cd210 con 0x7f1e04145250 2024-03-08T06:06:14.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.365+0000 7f1e0a600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e041457b0 con 0x7f1e040717d0 2024-03-08T06:06:14.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.366+0000 7f1e03e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04153280 0x7f1e0413e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:14.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.366+0000 7f1e03e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04153280 0x7f1e0413e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54078/0 (socket says 172.21.15.40:54078) 2024-03-08T06:06:14.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.366+0000 7f1e03e00640 1 -- 172.21.15.40:0/3622470476 learned_addr learned my addr 172.21.15.40:0/3622470476 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:14.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.366+0000 7f1e02a00640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e04145250 0x7f1e0413cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:14.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.366+0000 7f1e03400640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e040717d0 0x7f1e0415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:14.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.366+0000 7f1e03e00640 1 -- 172.21.15.40:0/3622470476 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e040717d0 msgr2=0x7f1e0415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:14.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.366+0000 7f1e03e00640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e040717d0 0x7f1e0415e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.366+0000 7f1e03e00640 1 -- 172.21.15.40:0/3622470476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e04145250 msgr2=0x7f1e0413cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:14.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.366+0000 7f1e03e00640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e04145250 0x7f1e0413cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.366+0000 7f1e03e00640 1 -- 172.21.15.40:0/3622470476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0413d290 con 0x7f1e04153280 2024-03-08T06:06:14.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.366+0000 7f1e03e00640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04153280 0x7f1e0413e720 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00c850 tx=0x7f1dfc00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:14.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.367+0000 7f1dfb400640 1 -- 172.21.15.40:0/3622470476 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc015070 con 0x7f1e04153280 2024-03-08T06:06:14.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.367+0000 7f1dfb400640 1 -- 172.21.15.40:0/3622470476 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc007830 con 0x7f1e04153280 2024-03-08T06:06:14.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.367+0000 7f1dfb400640 1 -- 172.21.15.40:0/3622470476 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc007c20 con 0x7f1e04153280 2024-03-08T06:06:14.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.367+0000 7f1e0a600640 1 -- 172.21.15.40:0/3622470476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0413f4f0 con 0x7f1e04153280 2024-03-08T06:06:14.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.367+0000 7f1e0a600640 1 -- 172.21.15.40:0/3622470476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0413fab0 con 0x7f1e04153280 2024-03-08T06:06:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.368+0000 7f1e0a600640 1 -- 172.21.15.40:0/3622470476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e04067cb0 con 0x7f1e04153280 2024-03-08T06:06:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.369+0000 7f1dfb400640 1 -- 172.21.15.40:0/3622470476 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1dfc018070 con 0x7f1e04153280 2024-03-08T06:06:14.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.370+0000 7f1dfb400640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1de006b4d0 0x7f1de006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:14.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.370+0000 7f1dfb400640 1 -- 172.21.15.40:0/3622470476 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1dfc08a1d0 con 0x7f1e04153280 2024-03-08T06:06:14.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.370+0000 7f1e03400640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1de006b4d0 0x7f1de006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:14.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.371+0000 7f1e03400640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1de006b4d0 0x7f1de006d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1df4007dc0 tx=0x7f1df40072f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:14.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.375+0000 7f1dfb400640 1 -- 172.21.15.40:0/3622470476 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1dfc058fb0 con 0x7f1e04153280 2024-03-08T06:06:14.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.590+0000 7f1e0a600640 1 -- 172.21.15.40:0/3622470476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1e0406b210 con 0x7f1e04153280 2024-03-08T06:06:14.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.591+0000 7f1dfb400640 1 -- 172.21.15.40:0/3622470476 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1dfc0588b0 con 0x7f1e04153280 2024-03-08T06:06:14.592 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.594+0000 7f1e0a600640 1 -- 172.21.15.40:0/3622470476 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1de006b4d0 msgr2=0x7f1de006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.594+0000 7f1e0a600640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1de006b4d0 0x7f1de006d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1df4007dc0 tx=0x7f1df40072f0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.594+0000 7f1e0a600640 1 -- 172.21.15.40:0/3622470476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04153280 msgr2=0x7f1e0413e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.594+0000 7f1e0a600640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04153280 0x7f1e0413e720 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00c850 tx=0x7f1dfc00cd10 comp rx=0 tx=0).stop 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.595+0000 7f1e0a600640 1 -- 172.21.15.40:0/3622470476 shutdown_connections 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.595+0000 7f1e0a600640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f1de006b4d0 0x7f1de006d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.595+0000 7f1e0a600640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e04153280 0x7f1e0413e720 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.595+0000 7f1e0a600640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e04145250 0x7f1e0413cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.595+0000 7f1e0a600640 1 --2- 172.21.15.40:0/3622470476 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e040717d0 0x7f1e0415e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.595+0000 7f1e0a600640 1 -- 172.21.15.40:0/3622470476 >> 172.21.15.40:0/3622470476 conn(0x7f1e0406ae30 msgr2=0x7f1e0406db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.595+0000 7f1e0a600640 1 -- 172.21.15.40:0/3622470476 shutdown_connections 2024-03-08T06:06:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:14.595+0000 7f1e0a600640 1 -- 172.21.15.40:0/3622470476 wait complete. 2024-03-08T06:06:14.615 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-08T06:06:14.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:14.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-08T06:06:15.072 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:06:15.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:15.072 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-08T06:06:15.116 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:06:15.116 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:06:15.175 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:06:15.175 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:06:15.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:06:15.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.444+0000 7f6c83e00640 1 Processor -- start 2024-03-08T06:06:15.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.444+0000 7f6c83e00640 1 -- start start 2024-03-08T06:06:15.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.445+0000 7f6c83e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6c8415a770 0x7f6c8415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.445+0000 7f6c83e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c840cd020 con 0x7f6c840cc310 2024-03-08T06:06:15.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.445+0000 7f6c83e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c840cd300 con 0x7f6c8406ea40 2024-03-08T06:06:15.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.445+0000 7f6c83e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c840ccdd0 con 0x7f6c8415a770 2024-03-08T06:06:15.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.445+0000 7f6c7a000640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6c840cc310 0x7f6c8406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35826/0 (socket says 172.21.15.40:35826) 2024-03-08T06:06:15.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.445+0000 7f6c7a000640 1 -- 172.21.15.40:0/3579712281 learned_addr learned my addr 172.21.15.40:0/3579712281 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82a00640 1 --2- 172.21.15.40:0/3579712281 >> v2:172.21.15.53:3300/0 conn(0x7f6c8415a770 0x7f6c8415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833185160 0 0) 0x7f6c840cd020 con 0x7f6c840cc310 2024-03-08T06:06:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c68003520 con 0x7f6c840cc310 2024-03-08T06:06:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3293650696 0 0) 0x7f6c68003520 con 0x7f6c840cc310 2024-03-08T06:06:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 >> v2:172.21.15.53:3300/0 conn(0x7f6c8415a770 msgr2=0x7f6c8415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 --2- 172.21.15.40:0/3579712281 >> v2:172.21.15.53:3300/0 conn(0x7f6c8415a770 0x7f6c8415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 >> v1:172.21.15.53:6789/0 conn(0x7f6c8406ea40 legacy=0x7f6c8415ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8415e580 con 0x7f6c840cc310 2024-03-08T06:06:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f6c6c001670 con 0x7f6c840cc310 2024-03-08T06:06:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 >> v1:172.21.15.40:6789/0 conn(0x7f6c840cc310 legacy=0x7f6c8406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 --2- 172.21.15.40:0/3579712281 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c68003a00 0x7f6c68003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 --2- 172.21.15.40:0/3579712281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 0x7f6c68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.446+0000 7f6c82000640 1 --2- 172.21.15.40:0/3579712281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c680051c0 0x7f6c6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c840ccdd0 con 0x7f6c68004800 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6800c0f0 con 0x7f6c680051c0 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6800c2f0 con 0x7f6c68003a00 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c7a000640 1 --2- 172.21.15.40:0/3579712281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 0x7f6c68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c83400640 1 --2- 172.21.15.40:0/3579712281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c680051c0 0x7f6c6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c82a00640 1 --2- 172.21.15.40:0/3579712281 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c68003a00 0x7f6c68003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c7a000640 1 -- 172.21.15.40:0/3579712281 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c68003a00 msgr2=0x7f6c68003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c7a000640 1 --2- 172.21.15.40:0/3579712281 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c68003a00 0x7f6c68003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c7a000640 1 -- 172.21.15.40:0/3579712281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c680051c0 msgr2=0x7f6c6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:15.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c7a000640 1 --2- 172.21.15.40:0/3579712281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c680051c0 0x7f6c6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c83400640 1 -- 172.21.15.40:0/3579712281 reap_dead start 2024-03-08T06:06:15.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.447+0000 7f6c7a000640 1 -- 172.21.15.40:0/3579712281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68003520 con 0x7f6c68004800 2024-03-08T06:06:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.449+0000 7f6c7a000640 1 --2- 172.21.15.40:0/3579712281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 0x7f6c68004c80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00c110 tx=0x7f6c68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc88932a63033ad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.449+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c007c40 con 0x7f6c68004800 2024-03-08T06:06:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.449+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c030070 con 0x7f6c68004800 2024-03-08T06:06:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.449+0000 7f6c82000640 1 -- 172.21.15.40:0/3579712281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c005d70 con 0x7f6c68004800 2024-03-08T06:06:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.450+0000 7f6c83e00640 1 -- 172.21.15.40:0/3579712281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 msgr2=0x7f6c68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.450+0000 7f6c83e00640 1 --2- 172.21.15.40:0/3579712281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 0x7f6c68004c80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00c110 tx=0x7f6c68005830 comp rx=0 tx=0).stop 2024-03-08T06:06:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.450+0000 7f6c83e00640 1 -- 172.21.15.40:0/3579712281 shutdown_connections 2024-03-08T06:06:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.450+0000 7f6c83e00640 1 --2- 172.21.15.40:0/3579712281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 0x7f6c68004c80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.450+0000 7f6c83e00640 1 -- 172.21.15.40:0/3579712281 >> 172.21.15.40:0/3579712281 conn(0x7f6c840b1850 msgr2=0x7f6c840b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.450+0000 7f6c83e00640 1 -- 172.21.15.40:0/3579712281 shutdown_connections 2024-03-08T06:06:15.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.450+0000 7f6c83e00640 1 -- 172.21.15.40:0/3579712281 wait complete. 2024-03-08T06:06:15.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.452+0000 7f6c83e00640 1 Processor -- start 2024-03-08T06:06:15.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.452+0000 7f6c83e00640 1 -- start start 2024-03-08T06:06:15.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.453+0000 7f6c83e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 0x7f6c84151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.453+0000 7f6c83e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8414ee00 0x7f6c8406ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.453+0000 7f6c83e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c8414f280 0x7f6c8406b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.453+0000 7f6c83e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c840cc790 con 0x7f6c68004800 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.453+0000 7f6c83e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8415e8f0 con 0x7f6c8414ee00 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.453+0000 7f6c83e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c840cc910 con 0x7f6c8414f280 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.453+0000 7f6c82a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 0x7f6c84151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.453+0000 7f6c82a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 0x7f6c84151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54098/0 (socket says 172.21.15.40:54098) 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.453+0000 7f6c82a00640 1 -- 172.21.15.40:0/250058351 learned_addr learned my addr 172.21.15.40:0/250058351 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.454+0000 7f6c82a00640 1 -- 172.21.15.40:0/250058351 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c8414f280 msgr2=0x7f6c8406b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.454+0000 7f6c82a00640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c8414f280 0x7f6c8406b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.454+0000 7f6c82a00640 1 -- 172.21.15.40:0/250058351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8414ee00 msgr2=0x7f6c8406ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.454+0000 7f6c7a000640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8414ee00 0x7f6c8406ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.454+0000 7f6c82a00640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8414ee00 0x7f6c8406ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.454+0000 7f6c82a00640 1 -- 172.21.15.40:0/250058351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8414f850 con 0x7f6c68004800 2024-03-08T06:06:15.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.454+0000 7f6c82a00640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 0x7f6c84151d10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6c74010790 tx=0x7f6c74010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:15.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.454+0000 7f6c7b400640 1 -- 172.21.15.40:0/250058351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400f040 con 0x7f6c68004800 2024-03-08T06:06:15.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.454+0000 7f6c7b400640 1 -- 172.21.15.40:0/250058351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7400eb60 con 0x7f6c68004800 2024-03-08T06:06:15.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.454+0000 7f6c7b400640 1 -- 172.21.15.40:0/250058351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74010ea0 con 0x7f6c68004800 2024-03-08T06:06:15.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.454+0000 7f6c83e00640 1 -- 172.21.15.40:0/250058351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c84151230 con 0x7f6c68004800 2024-03-08T06:06:15.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.455+0000 7f6c83e00640 1 -- 172.21.15.40:0/250058351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c841517f0 con 0x7f6c68004800 2024-03-08T06:06:15.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.456+0000 7f6c83e00640 1 -- 172.21.15.40:0/250058351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8406d7d0 con 0x7f6c68004800 2024-03-08T06:06:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.458+0000 7f6c7b400640 1 -- 172.21.15.40:0/250058351 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c7401c400 con 0x7f6c68004800 2024-03-08T06:06:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.458+0000 7f6c7b400640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6c6006b4d0 0x7f6c6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.458+0000 7f6c7b400640 1 -- 172.21.15.40:0/250058351 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c7408a880 con 0x7f6c68004800 2024-03-08T06:06:15.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.458+0000 7f6c7a000640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6c6006b4d0 0x7f6c6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.459+0000 7f6c7a000640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6c6006b4d0 0x7f6c6006d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c034000 tx=0x7f6c6c039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:15.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.463+0000 7f6c7b400640 1 -- 172.21.15.40:0/250058351 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c8406d7d0 con 0x7f6c68004800 2024-03-08T06:06:15.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.662+0000 7f6c83e00640 1 -- 172.21.15.40:0/250058351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c84151020 con 0x7f6c68004800 2024-03-08T06:06:15.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.665+0000 7f6c7b400640 1 -- 172.21.15.40:0/250058351 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16585 (secure 0 0 0) 0x7f6c74018030 con 0x7f6c68004800 2024-03-08T06:06:15.667 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:15.667 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:05:12.978712+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:05:09.544904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T06:05:10.468274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:06:15.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 -- 172.21.15.40:0/250058351 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6c6006b4d0 msgr2=0x7f6c6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:15.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6c6006b4d0 0x7f6c6006d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c034000 tx=0x7f6c6c039040 comp rx=0 tx=0).stop 2024-03-08T06:06:15.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 -- 172.21.15.40:0/250058351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 msgr2=0x7f6c84151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:15.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 0x7f6c84151d10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6c74010790 tx=0x7f6c74010c50 comp rx=0 tx=0).stop 2024-03-08T06:06:15.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 -- 172.21.15.40:0/250058351 shutdown_connections 2024-03-08T06:06:15.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f6c6006b4d0 0x7f6c6006d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c8414f280 0x7f6c8406b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8414ee00 0x7f6c8406ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 --2- 172.21.15.40:0/250058351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c68004800 0x7f6c84151d10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 -- 172.21.15.40:0/250058351 >> 172.21.15.40:0/250058351 conn(0x7f6c840b1850 msgr2=0x7f6c8406dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:15.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 -- 172.21.15.40:0/250058351 shutdown_connections 2024-03-08T06:06:15.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.668+0000 7f6c83e00640 1 -- 172.21.15.40:0/250058351 wait complete. 2024-03-08T06:06:15.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:05:09.544904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:05:10.468274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:06:15.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:06:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.910+0000 7f03f2200640 1 Processor -- start 2024-03-08T06:06:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.910+0000 7f03f2200640 1 -- start start 2024-03-08T06:06:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.912+0000 7f03f2200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03ec15bfb0 0x7f03ec0716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.912+0000 7f03f2200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f03ec071c20 0x7f03ec0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.912+0000 7f03f2200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ec15c460 con 0x7f03ec145500 2024-03-08T06:06:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.912+0000 7f03f2200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ec072050 con 0x7f03ec071c20 2024-03-08T06:06:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.912+0000 7f03f2200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec15c740 con 0x7f03ec15bfb0 2024-03-08T06:06:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.912+0000 7f03f1800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f03ec071c20 0x7f03ec0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.912+0000 7f03ebe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03ec15bfb0 0x7f03ec0716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.912+0000 7f03f1800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f03ec071c20 0x7f03ec0cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:56656/0 (socket says 172.21.15.40:56656) 2024-03-08T06:06:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.912+0000 7f03f1800640 1 -- 172.21.15.40:0/295631561 learned_addr learned my addr 172.21.15.40:0/295631561 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:15.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.913+0000 7f03f1800640 1 -- 172.21.15.40:0/295631561 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03ec15bfb0 msgr2=0x7f03ec0716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:15.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.913+0000 7f03f1800640 1 --2- 172.21.15.40:0/295631561 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03ec15bfb0 0x7f03ec0716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.913+0000 7f03f1800640 1 -- 172.21.15.40:0/295631561 >> v1:172.21.15.53:6789/0 conn(0x7f03ec145500 legacy=0x7f03ec06ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:15.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.913+0000 7f03f1800640 1 -- 172.21.15.40:0/295631561 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec15ca20 con 0x7f03ec071c20 2024-03-08T06:06:15.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.915+0000 7f03f1800640 1 --2- 172.21.15.40:0/295631561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ec071c20 0x7f03ec0cc500 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009e50 tx=0x7f03e402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9324655f28674cef server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:15.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.915+0000 7f03eb400640 1 -- 172.21.15.40:0/295631561 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e402c9e0 con 0x7f03ec071c20 2024-03-08T06:06:15.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.915+0000 7f03eb400640 1 -- 172.21.15.40:0/295631561 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e402cb80 con 0x7f03ec071c20 2024-03-08T06:06:15.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.915+0000 7f03eb400640 1 -- 172.21.15.40:0/295631561 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40343f0 con 0x7f03ec071c20 2024-03-08T06:06:15.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.916+0000 7f03f2200640 1 -- 172.21.15.40:0/295631561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ec071c20 msgr2=0x7f03ec0cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:15.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.916+0000 7f03f2200640 1 --2- 172.21.15.40:0/295631561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ec071c20 0x7f03ec0cc500 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009e50 tx=0x7f03e402be40 comp rx=0 tx=0).stop 2024-03-08T06:06:15.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.916+0000 7f03f2200640 1 -- 172.21.15.40:0/295631561 shutdown_connections 2024-03-08T06:06:15.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.916+0000 7f03f2200640 1 --2- 172.21.15.40:0/295631561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ec071c20 0x7f03ec0cc500 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.916+0000 7f03f2200640 1 --2- 172.21.15.40:0/295631561 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03ec15bfb0 0x7f03ec0716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.916+0000 7f03f2200640 1 -- 172.21.15.40:0/295631561 >> 172.21.15.40:0/295631561 conn(0x7f03ec0b1850 msgr2=0x7f03ec0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:15.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.916+0000 7f03f2200640 1 -- 172.21.15.40:0/295631561 shutdown_connections 2024-03-08T06:06:15.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.916+0000 7f03f2200640 1 -- 172.21.15.40:0/295631561 wait complete. 2024-03-08T06:06:15.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.918+0000 7f03f2200640 1 Processor -- start 2024-03-08T06:06:15.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.919+0000 7f03f2200640 1 -- start start 2024-03-08T06:06:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f2200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ec071c20 0x7f03ec13a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f2200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ec145500 0x7f03ec13f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f2200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03ec15bfb0 0x7f03ec13c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f2200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec0ccbc0 con 0x7f03ec071c20 2024-03-08T06:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f2200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec0cca40 con 0x7f03ec145500 2024-03-08T06:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f2200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec0ccd40 con 0x7f03ec15bfb0 2024-03-08T06:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f0e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ec071c20 0x7f03ec13a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f0e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ec071c20 0x7f03ec13a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54110/0 (socket says 172.21.15.40:54110) 2024-03-08T06:06:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f0e00640 1 -- 172.21.15.40:0/691384742 learned_addr learned my addr 172.21.15.40:0/691384742 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03ebe00640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ec145500 0x7f03ec13f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f1800640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03ec15bfb0 0x7f03ec13c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f0e00640 1 -- 172.21.15.40:0/691384742 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03ec15bfb0 msgr2=0x7f03ec13c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f0e00640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03ec15bfb0 0x7f03ec13c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f0e00640 1 -- 172.21.15.40:0/691384742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ec145500 msgr2=0x7f03ec13f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f0e00640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ec145500 0x7f03ec13f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f0e00640 1 -- 172.21.15.40:0/691384742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec13aeb0 con 0x7f03ec071c20 2024-03-08T06:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03ebe00640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ec145500 0x7f03ec13f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f0e00640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ec071c20 0x7f03ec13a970 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00c430 tx=0x7f03dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03e8c00640 1 -- 172.21.15.40:0/691384742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc011790 con 0x7f03ec071c20 2024-03-08T06:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03e8c00640 1 -- 172.21.15.40:0/691384742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc017070 con 0x7f03ec071c20 2024-03-08T06:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03e8c00640 1 -- 172.21.15.40:0/691384742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc011bb0 con 0x7f03ec071c20 2024-03-08T06:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.921+0000 7f03f2200640 1 -- 172.21.15.40:0/691384742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec13b160 con 0x7f03ec071c20 2024-03-08T06:06:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.922+0000 7f03f2200640 1 -- 172.21.15.40:0/691384742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec13b720 con 0x7f03ec071c20 2024-03-08T06:06:15.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.923+0000 7f03f2200640 1 -- 172.21.15.40:0/691384742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ec075090 con 0x7f03ec071c20 2024-03-08T06:06:15.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.924+0000 7f03e8c00640 1 -- 172.21.15.40:0/691384742 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03dc01c460 con 0x7f03ec071c20 2024-03-08T06:06:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.925+0000 7f03e8c00640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f03cc06b4d0 0x7f03cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.925+0000 7f03e8c00640 1 -- 172.21.15.40:0/691384742 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03dc08a0f0 con 0x7f03ec071c20 2024-03-08T06:06:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.926+0000 7f03ebe00640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f03cc06b4d0 0x7f03cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.926+0000 7f03ebe00640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f03cc06b4d0 0x7f03cc06d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f03ec1f71d0 tx=0x7f03d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:15.930+0000 7f03e8c00640 1 -- 172.21.15.40:0/691384742 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03dc058f60 con 0x7f03ec071c20 2024-03-08T06:06:16.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.134+0000 7f03f2200640 1 -- 172.21.15.40:0/691384742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03ec1397d0 con 0x7f03ec071c20 2024-03-08T06:06:16.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.135+0000 7f03e8c00640 1 -- 172.21.15.40:0/691384742 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03dc058860 con 0x7f03ec071c20 2024-03-08T06:06:16.136 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:06:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 -- 172.21.15.40:0/691384742 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f03cc06b4d0 msgr2=0x7f03cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f03cc06b4d0 0x7f03cc06d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f03ec1f71d0 tx=0x7f03d8008040 comp rx=0 tx=0).stop 2024-03-08T06:06:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 -- 172.21.15.40:0/691384742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ec071c20 msgr2=0x7f03ec13a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ec071c20 0x7f03ec13a970 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00c430 tx=0x7f03dc00c8f0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 -- 172.21.15.40:0/691384742 shutdown_connections 2024-03-08T06:06:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f03cc06b4d0 0x7f03cc06d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03ec15bfb0 0x7f03ec13c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03ec145500 0x7f03ec13f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 --2- 172.21.15.40:0/691384742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ec071c20 0x7f03ec13a970 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 -- 172.21.15.40:0/691384742 >> 172.21.15.40:0/691384742 conn(0x7f03ec0b1850 msgr2=0x7f03ec0b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 -- 172.21.15.40:0/691384742 shutdown_connections 2024-03-08T06:06:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.138+0000 7f03f2200640 1 -- 172.21.15.40:0/691384742 wait complete. 2024-03-08T06:06:16.158 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:06:16.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.385+0000 7fc2fe400640 1 Processor -- start 2024-03-08T06:06:16.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.385+0000 7fc2fe400640 1 -- start start 2024-03-08T06:06:16.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.387+0000 7fc2fe400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2f815bc50 0x7fc2f80d0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.387+0000 7fc2fe400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f80cf7a0 0x7fc2f8099e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.387+0000 7fc2fe400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f80d0b60 con 0x7fc2f815c030 2024-03-08T06:06:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.387+0000 7fc2fe400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f80cca10 con 0x7fc2f815bc50 2024-03-08T06:06:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.387+0000 7fc2fe400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80cccf0 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.388+0000 7fc2f7e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f80cf7a0 0x7fc2f8099e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.388+0000 7fc2f7400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2f815bc50 0x7fc2f80d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.388+0000 7fc2f7e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f80cf7a0 0x7fc2f8099e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:42496/0 (socket says 172.21.15.40:42496) 2024-03-08T06:06:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.388+0000 7fc2f7e00640 1 -- 172.21.15.40:0/1074541756 learned_addr learned my addr 172.21.15.40:0/1074541756 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.388+0000 7fc2f7e00640 1 -- 172.21.15.40:0/1074541756 >> v2:172.21.15.53:3300/0 conn(0x7fc2f815bc50 msgr2=0x7fc2f80d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.388+0000 7fc2f7e00640 1 --2- 172.21.15.40:0/1074541756 >> v2:172.21.15.53:3300/0 conn(0x7fc2f815bc50 0x7fc2f80d0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.388+0000 7fc2f7e00640 1 -- 172.21.15.40:0/1074541756 >> v1:172.21.15.53:6789/0 conn(0x7fc2f815c030 legacy=0x7fc2f80cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.388+0000 7fc2f7e00640 1 -- 172.21.15.40:0/1074541756 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f80ccfd0 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.390+0000 7fc2f7e00640 1 --2- 172.21.15.40:0/1074541756 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f80cf7a0 0x7fc2f8099e20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009e50 tx=0x7fc2f002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb04afe95599fa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:16.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.390+0000 7fc2f6000640 1 -- 172.21.15.40:0/1074541756 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f002cb00 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.391+0000 7fc2f6000640 1 -- 172.21.15.40:0/1074541756 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f002cca0 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.391+0000 7fc2f6000640 1 -- 172.21.15.40:0/1074541756 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0034610 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.391+0000 7fc2fe400640 1 -- 172.21.15.40:0/1074541756 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f80cf7a0 msgr2=0x7fc2f8099e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.391+0000 7fc2fe400640 1 --2- 172.21.15.40:0/1074541756 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f80cf7a0 0x7fc2f8099e20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009e50 tx=0x7fc2f002bf60 comp rx=0 tx=0).stop 2024-03-08T06:06:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.391+0000 7fc2fe400640 1 -- 172.21.15.40:0/1074541756 shutdown_connections 2024-03-08T06:06:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.391+0000 7fc2fe400640 1 --2- 172.21.15.40:0/1074541756 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f80cf7a0 0x7fc2f8099e20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.391+0000 7fc2fe400640 1 --2- 172.21.15.40:0/1074541756 >> v2:172.21.15.53:3300/0 conn(0x7fc2f815bc50 0x7fc2f80d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.391+0000 7fc2fe400640 1 -- 172.21.15.40:0/1074541756 >> 172.21.15.40:0/1074541756 conn(0x7fc2f80b3650 msgr2=0x7fc2f80b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.391+0000 7fc2fe400640 1 -- 172.21.15.40:0/1074541756 shutdown_connections 2024-03-08T06:06:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.391+0000 7fc2fe400640 1 -- 172.21.15.40:0/1074541756 wait complete. 2024-03-08T06:06:16.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.393+0000 7fc2fe400640 1 Processor -- start 2024-03-08T06:06:16.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.393+0000 7fc2fe400640 1 -- start start 2024-03-08T06:06:16.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.394+0000 7fc2fe400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f80cf7a0 0x7fc2f80b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.394+0000 7fc2fe400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815bc50 0x7fc2f8150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.394+0000 7fc2fe400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f815c030 0x7fc2f80c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.394+0000 7fc2fe400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80cc670 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.394+0000 7fc2fe400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80cd590 con 0x7fc2f815bc50 2024-03-08T06:06:16.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.394+0000 7fc2fe400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80cc7f0 con 0x7fc2f815c030 2024-03-08T06:06:16.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.395+0000 7fc2f7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f80cf7a0 0x7fc2f80b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.395+0000 7fc2f7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f80cf7a0 0x7fc2f80b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54124/0 (socket says 172.21.15.40:54124) 2024-03-08T06:06:16.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.395+0000 7fc2f7400640 1 -- 172.21.15.40:0/2021444874 learned_addr learned my addr 172.21.15.40:0/2021444874 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:16.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.395+0000 7fc2f6a00640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815bc50 0x7fc2f8150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.395+0000 7fc2f7e00640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f815c030 0x7fc2f80c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.395+0000 7fc2f7400640 1 -- 172.21.15.40:0/2021444874 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f815c030 msgr2=0x7fc2f80c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.395+0000 7fc2f7400640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f815c030 0x7fc2f80c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.395+0000 7fc2f7400640 1 -- 172.21.15.40:0/2021444874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815bc50 msgr2=0x7fc2f8150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.395+0000 7fc2f7400640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815bc50 0x7fc2f8150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.395+0000 7fc2f7400640 1 -- 172.21.15.40:0/2021444874 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f81f2120 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.395+0000 7fc2f7e00640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f815c030 0x7fc2f80c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:16.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.396+0000 7fc2f7400640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f80cf7a0 0x7fc2f80b56a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800e9e0 tx=0x7fc2e800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:16.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.396+0000 7fc2ef400640 1 -- 172.21.15.40:0/2021444874 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8017070 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.396+0000 7fc2ef400640 1 -- 172.21.15.40:0/2021444874 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e800c9d0 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.396+0000 7fc2fe400640 1 -- 172.21.15.40:0/2021444874 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f80ca020 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.396+0000 7fc2ef400640 1 -- 172.21.15.40:0/2021444874 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e800cdc0 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.396+0000 7fc2fe400640 1 -- 172.21.15.40:0/2021444874 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f80ca5e0 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.397+0000 7fc2fe400640 1 -- 172.21.15.40:0/2021444874 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f80683c0 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.398+0000 7fc2ef400640 1 -- 172.21.15.40:0/2021444874 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2e801a070 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.399+0000 7fc2ef400640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc2d806b4d0 0x7fc2d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.400+0000 7fc2ef400640 1 -- 172.21.15.40:0/2021444874 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2e808a250 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.400+0000 7fc2f6a00640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc2d806b4d0 0x7fc2d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.401+0000 7fc2f6a00640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc2d806b4d0 0x7fc2d806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80b2ab0 tx=0x7fc2e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:16.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.404+0000 7fc2ef400640 1 -- 172.21.15.40:0/2021444874 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2e80590c0 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.603+0000 7fc2fe400640 1 -- 172.21.15.40:0/2021444874 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2f80b44f0 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.604+0000 7fc2ef400640 1 -- 172.21.15.40:0/2021444874 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2e80589c0 con 0x7fc2f80cf7a0 2024-03-08T06:06:16.605 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:06:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 -- 172.21.15.40:0/2021444874 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc2d806b4d0 msgr2=0x7fc2d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc2d806b4d0 0x7fc2d806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80b2ab0 tx=0x7fc2e4009040 comp rx=0 tx=0).stop 2024-03-08T06:06:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 -- 172.21.15.40:0/2021444874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f80cf7a0 msgr2=0x7fc2f80b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f80cf7a0 0x7fc2f80b56a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800e9e0 tx=0x7fc2e800eea0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 -- 172.21.15.40:0/2021444874 shutdown_connections 2024-03-08T06:06:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc2d806b4d0 0x7fc2d806d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc2f815c030 0x7fc2f80c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f815bc50 0x7fc2f8150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 --2- 172.21.15.40:0/2021444874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2f80cf7a0 0x7fc2f80b56a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 -- 172.21.15.40:0/2021444874 >> 172.21.15.40:0/2021444874 conn(0x7fc2f80b3650 msgr2=0x7fc2f815d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 -- 172.21.15.40:0/2021444874 shutdown_connections 2024-03-08T06:06:16.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.607+0000 7fc2fe400640 1 -- 172.21.15.40:0/2021444874 wait complete. 2024-03-08T06:06:16.626 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:06:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.851+0000 7f4580800640 1 Processor -- start 2024-03-08T06:06:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.852+0000 7f4580800640 1 -- start start 2024-03-08T06:06:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.853+0000 7f4580800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4578145590 0x7f45780cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.853+0000 7f4580800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f45780ccc70 0x7f45780cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.853+0000 7f4580800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4578071990 con 0x7f457815f7f0 2024-03-08T06:06:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.853+0000 7f4580800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45780cd5f0 con 0x7f45780ccc70 2024-03-08T06:06:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.854+0000 7f4580800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4578071c70 con 0x7f4578145590 2024-03-08T06:06:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.854+0000 7f457d400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f45780ccc70 0x7f45780cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.854+0000 7f457de00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4578145590 0x7f45780cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.854+0000 7f457de00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4578145590 0x7f45780cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:42508/0 (socket says 172.21.15.40:42508) 2024-03-08T06:06:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.854+0000 7f457de00640 1 -- 172.21.15.40:0/2066377376 learned_addr learned my addr 172.21.15.40:0/2066377376 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:16.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.855+0000 7f457de00640 1 -- 172.21.15.40:0/2066377376 >> v2:172.21.15.53:3300/0 conn(0x7f45780ccc70 msgr2=0x7f45780cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:16.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.855+0000 7f457de00640 1 --2- 172.21.15.40:0/2066377376 >> v2:172.21.15.53:3300/0 conn(0x7f45780ccc70 0x7f45780cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.855+0000 7f457de00640 1 -- 172.21.15.40:0/2066377376 >> v1:172.21.15.53:6789/0 conn(0x7f457815f7f0 legacy=0x7f457815fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.855+0000 7f457de00640 1 -- 172.21.15.40:0/2066377376 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4578071f50 con 0x7f4578145590 2024-03-08T06:06:16.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.856+0000 7f457de00640 1 --2- 172.21.15.40:0/2066377376 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4578145590 0x7f45780cf1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f456c009870 tx=0x7f456c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5620ad999e83d35 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.857+0000 7f457ca00640 1 -- 172.21.15.40:0/2066377376 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c034070 con 0x7f4578145590 2024-03-08T06:06:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.857+0000 7f457ca00640 1 -- 172.21.15.40:0/2066377376 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c030840 con 0x7f4578145590 2024-03-08T06:06:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.857+0000 7f457ca00640 1 -- 172.21.15.40:0/2066377376 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c030c10 con 0x7f4578145590 2024-03-08T06:06:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.857+0000 7f4580800640 1 -- 172.21.15.40:0/2066377376 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4578145590 msgr2=0x7f45780cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.857+0000 7f4580800640 1 --2- 172.21.15.40:0/2066377376 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4578145590 0x7f45780cf1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f456c009870 tx=0x7f456c02b980 comp rx=0 tx=0).stop 2024-03-08T06:06:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.858+0000 7f4580800640 1 -- 172.21.15.40:0/2066377376 shutdown_connections 2024-03-08T06:06:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.858+0000 7f4580800640 1 --2- 172.21.15.40:0/2066377376 >> v2:172.21.15.53:3300/0 conn(0x7f45780ccc70 0x7f45780cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.858+0000 7f4580800640 1 --2- 172.21.15.40:0/2066377376 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4578145590 0x7f45780cf1a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.858+0000 7f4580800640 1 -- 172.21.15.40:0/2066377376 >> 172.21.15.40:0/2066377376 conn(0x7f45780cfaa0 msgr2=0x7f45780cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.858+0000 7f4580800640 1 -- 172.21.15.40:0/2066377376 shutdown_connections 2024-03-08T06:06:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.858+0000 7f4580800640 1 -- 172.21.15.40:0/2066377376 wait complete. 2024-03-08T06:06:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.859+0000 7f4580800640 1 Processor -- start 2024-03-08T06:06:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.860+0000 7f4580800640 1 -- start start 2024-03-08T06:06:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.861+0000 7f4580800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f45780ccc70 0x7f45781395a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.861+0000 7f4580800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4578145590 0x7f45781424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.861+0000 7f4580800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f457815f7f0 0x7f457813b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.861+0000 7f4580800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4578145970 con 0x7f457815f7f0 2024-03-08T06:06:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.861+0000 7f4580800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4578072510 con 0x7f4578145590 2024-03-08T06:06:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.861+0000 7f4580800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45780cd960 con 0x7f45780ccc70 2024-03-08T06:06:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.861+0000 7f457e800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f457815f7f0 0x7f457813b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.861+0000 7f457e800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f457815f7f0 0x7f457813b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39676/0 (socket says 172.21.15.40:39676) 2024-03-08T06:06:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.861+0000 7f457e800640 1 -- 172.21.15.40:0/886812179 learned_addr learned my addr 172.21.15.40:0/886812179 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.861+0000 7f457d400640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4578145590 0x7f45781424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.861+0000 7f457de00640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f45780ccc70 0x7f45781395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.862+0000 7f457e800640 1 -- 172.21.15.40:0/886812179 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f45780ccc70 msgr2=0x7f45781395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.862+0000 7f457e800640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f45780ccc70 0x7f45781395a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.862+0000 7f457e800640 1 -- 172.21.15.40:0/886812179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4578145590 msgr2=0x7f45781424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.862+0000 7f457e800640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4578145590 0x7f45781424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.862+0000 7f457e800640 1 -- 172.21.15.40:0/886812179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457815f1e0 con 0x7f457815f7f0 2024-03-08T06:06:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.862+0000 7f457d400640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4578145590 0x7f45781424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:06:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.862+0000 7f457e800640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f457815f7f0 0x7f457813b770 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f457400c7d0 tx=0x7f457400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.862+0000 7f4572000640 1 -- 172.21.15.40:0/886812179 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574003b70 con 0x7f457815f7f0 2024-03-08T06:06:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.862+0000 7f4572000640 1 -- 172.21.15.40:0/886812179 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4574003d30 con 0x7f457815f7f0 2024-03-08T06:06:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.862+0000 7f4572000640 1 -- 172.21.15.40:0/886812179 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457400aa90 con 0x7f457815f7f0 2024-03-08T06:06:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.862+0000 7f4580800640 1 -- 172.21.15.40:0/886812179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457815f4c0 con 0x7f457815f7f0 2024-03-08T06:06:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.863+0000 7f4580800640 1 -- 172.21.15.40:0/886812179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457806f380 con 0x7f457815f7f0 2024-03-08T06:06:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.864+0000 7f4580800640 1 -- 172.21.15.40:0/886812179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4578067cb0 con 0x7f457815f7f0 2024-03-08T06:06:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.865+0000 7f4572000640 1 -- 172.21.15.40:0/886812179 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f457400ac30 con 0x7f457815f7f0 2024-03-08T06:06:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.865+0000 7f4572000640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f455c06b4d0 0x7f455c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.866+0000 7f4572000640 1 -- 172.21.15.40:0/886812179 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f457408a260 con 0x7f457815f7f0 2024-03-08T06:06:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.866+0000 7f457de00640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f455c06b4d0 0x7f455c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.867+0000 7f457de00640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f455c06b4d0 0x7f455c06d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f456c036000 tx=0x7f456c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:16.871+0000 7f4572000640 1 -- 172.21.15.40:0/886812179 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4574059040 con 0x7f457815f7f0 2024-03-08T06:06:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.071+0000 7f4580800640 1 -- 172.21.15.40:0/886812179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45781565f0 con 0x7f457815f7f0 2024-03-08T06:06:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.072+0000 7f4572000640 1 -- 172.21.15.40:0/886812179 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4574058940 con 0x7f457815f7f0 2024-03-08T06:06:17.073 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:06:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.075+0000 7f4580800640 1 -- 172.21.15.40:0/886812179 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f455c06b4d0 msgr2=0x7f455c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.075+0000 7f4580800640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f455c06b4d0 0x7f455c06d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f456c036000 tx=0x7f456c03b040 comp rx=0 tx=0).stop 2024-03-08T06:06:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.075+0000 7f4580800640 1 -- 172.21.15.40:0/886812179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f457815f7f0 msgr2=0x7f457813b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.075+0000 7f4580800640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f457815f7f0 0x7f457813b770 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f457400c7d0 tx=0x7f457400cc90 comp rx=0 tx=0).stop 2024-03-08T06:06:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.075+0000 7f4580800640 1 -- 172.21.15.40:0/886812179 shutdown_connections 2024-03-08T06:06:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.075+0000 7f4580800640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f455c06b4d0 0x7f455c06d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.075+0000 7f4580800640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f457815f7f0 0x7f457813b770 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.075+0000 7f4580800640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4578145590 0x7f45781424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.075+0000 7f4580800640 1 --2- 172.21.15.40:0/886812179 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f45780ccc70 0x7f45781395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.075+0000 7f4580800640 1 -- 172.21.15.40:0/886812179 >> 172.21.15.40:0/886812179 conn(0x7f45780cfaa0 msgr2=0x7f457815fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:17.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.076+0000 7f4580800640 1 -- 172.21.15.40:0/886812179 shutdown_connections 2024-03-08T06:06:17.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.076+0000 7f4580800640 1 -- 172.21.15.40:0/886812179 wait complete. 2024-03-08T06:06:17.096 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T06:06:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.323+0000 7fc660400640 1 Processor -- start 2024-03-08T06:06:17.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.323+0000 7fc660400640 1 -- start start 2024-03-08T06:06:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.325+0000 7fc660400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc658147480 0x7fc65806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.325+0000 7fc660400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc658162330 con 0x7fc658146f30 2024-03-08T06:06:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.325+0000 7fc660400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6581625c0 con 0x7fc658161840 2024-03-08T06:06:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.325+0000 7fc660400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6580b3520 con 0x7fc658147480 2024-03-08T06:06:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.325+0000 7fc65d000640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc658146f30 0x7fc658161c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41962/0 (socket says 172.21.15.40:41962) 2024-03-08T06:06:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.326+0000 7fc65d000640 1 -- 172.21.15.40:0/1954671280 learned_addr learned my addr 172.21.15.40:0/1954671280 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.326+0000 7fc65e400640 1 --2- 172.21.15.40:0/1954671280 >> v2:172.21.15.40:3300/0 conn(0x7fc658147480 0x7fc65806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.326+0000 7fc657e00640 1 -- 172.21.15.40:0/1954671280 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919507860 0 0) 0x7fc658162330 con 0x7fc658146f30 2024-03-08T06:06:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.326+0000 7fc657e00640 1 -- 172.21.15.40:0/1954671280 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc63c003520 con 0x7fc658146f30 2024-03-08T06:06:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.326+0000 7fc65e400640 1 -- 172.21.15.40:0/1954671280 >> v1:172.21.15.53:6789/0 conn(0x7fc658161840 legacy=0x7fc6580b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.326+0000 7fc65e400640 1 -- 172.21.15.40:0/1954671280 >> v1:172.21.15.40:6789/0 conn(0x7fc658146f30 legacy=0x7fc658161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.326+0000 7fc65e400640 1 -- 172.21.15.40:0/1954671280 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6580b4960 con 0x7fc658147480 2024-03-08T06:06:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.327+0000 7fc65e400640 1 --2- 172.21.15.40:0/1954671280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc658147480 0x7fc65806d440 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc6500097a0 tx=0x7fc65002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7125051aa68ab37 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.328+0000 7fc657e00640 1 -- 172.21.15.40:0/1954671280 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650034070 con 0x7fc658147480 2024-03-08T06:06:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.328+0000 7fc657e00640 1 -- 172.21.15.40:0/1954671280 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc650030610 con 0x7fc658147480 2024-03-08T06:06:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.328+0000 7fc657e00640 1 -- 172.21.15.40:0/1954671280 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6500309e0 con 0x7fc658147480 2024-03-08T06:06:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.328+0000 7fc660400640 1 -- 172.21.15.40:0/1954671280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc658147480 msgr2=0x7fc65806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.328+0000 7fc660400640 1 --2- 172.21.15.40:0/1954671280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc658147480 0x7fc65806d440 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc6500097a0 tx=0x7fc65002b960 comp rx=0 tx=0).stop 2024-03-08T06:06:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.328+0000 7fc660400640 1 -- 172.21.15.40:0/1954671280 shutdown_connections 2024-03-08T06:06:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.328+0000 7fc660400640 1 --2- 172.21.15.40:0/1954671280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc658147480 0x7fc65806d440 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.328+0000 7fc660400640 1 -- 172.21.15.40:0/1954671280 >> 172.21.15.40:0/1954671280 conn(0x7fc6580b16e0 msgr2=0x7fc6580b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.329+0000 7fc660400640 1 -- 172.21.15.40:0/1954671280 shutdown_connections 2024-03-08T06:06:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.329+0000 7fc660400640 1 -- 172.21.15.40:0/1954671280 wait complete. 2024-03-08T06:06:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.330+0000 7fc660400640 1 Processor -- start 2024-03-08T06:06:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.330+0000 7fc660400640 1 -- start start 2024-03-08T06:06:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.331+0000 7fc660400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc658146f30 0x7fc6581450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.331+0000 7fc660400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc658147480 0x7fc658143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.331+0000 7fc660400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc658161840 0x7fc658144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.331+0000 7fc660400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65806d980 con 0x7fc658147480 2024-03-08T06:06:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.331+0000 7fc660400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6580b4c40 con 0x7fc658161840 2024-03-08T06:06:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.331+0000 7fc660400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65806db00 con 0x7fc658146f30 2024-03-08T06:06:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.332+0000 7fc65d000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc658147480 0x7fc658143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.332+0000 7fc65d000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc658147480 0x7fc658143bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39692/0 (socket says 172.21.15.40:39692) 2024-03-08T06:06:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.332+0000 7fc65d000640 1 -- 172.21.15.40:0/1714910369 learned_addr learned my addr 172.21.15.40:0/1714910369 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:17.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.332+0000 7fc65e400640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc658161840 0x7fc658144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.332+0000 7fc65da00640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc658146f30 0x7fc6581450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.332+0000 7fc65d000640 1 -- 172.21.15.40:0/1714910369 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc658146f30 msgr2=0x7fc6581450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.332+0000 7fc65d000640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc658146f30 0x7fc6581450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.332+0000 7fc65d000640 1 -- 172.21.15.40:0/1714910369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc658161840 msgr2=0x7fc658144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.332+0000 7fc65d000640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc658161840 0x7fc658144110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.332+0000 7fc65d000640 1 -- 172.21.15.40:0/1714910369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc658144740 con 0x7fc658147480 2024-03-08T06:06:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.333+0000 7fc65d000640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc658147480 0x7fc658143bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc64400c500 tx=0x7fc64400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.333+0000 7fc655600640 1 -- 172.21.15.40:0/1714910369 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644017070 con 0x7fc658147480 2024-03-08T06:06:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.333+0000 7fc655600640 1 -- 172.21.15.40:0/1714910369 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644005ce0 con 0x7fc658147480 2024-03-08T06:06:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.333+0000 7fc660400640 1 -- 172.21.15.40:0/1714910369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65813d5b0 con 0x7fc658147480 2024-03-08T06:06:17.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.333+0000 7fc655600640 1 -- 172.21.15.40:0/1714910369 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644007330 con 0x7fc658147480 2024-03-08T06:06:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.333+0000 7fc660400640 1 -- 172.21.15.40:0/1714910369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65813de50 con 0x7fc658147480 2024-03-08T06:06:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.335+0000 7fc660400640 1 -- 172.21.15.40:0/1714910369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65813d9a0 con 0x7fc658147480 2024-03-08T06:06:17.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.336+0000 7fc655600640 1 -- 172.21.15.40:0/1714910369 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc64401a070 con 0x7fc658147480 2024-03-08T06:06:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.337+0000 7fc655600640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.337+0000 7fc655600640 1 -- 172.21.15.40:0/1714910369 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6440896d0 con 0x7fc658147480 2024-03-08T06:06:17.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.337+0000 7fc65da00640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:17.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.338+0000 7fc65da00640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc63806b4d0 0x7fc63806d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc648001c80 tx=0x7fc648009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:17.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.342+0000 7fc655600640 1 -- 172.21.15.40:0/1714910369 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc644058600 con 0x7fc658147480 2024-03-08T06:06:17.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.555+0000 7fc660400640 1 -- 172.21.15.40:0/1714910369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc658067cb0 con 0x7fc658147480 2024-03-08T06:06:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.556+0000 7fc655600640 1 -- 172.21.15.40:0/1714910369 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc644057f00 con 0x7fc658147480 2024-03-08T06:06:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-03-08T06:06:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.559+0000 7fc660400640 1 -- 172.21.15.40:0/1714910369 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc63806b4d0 msgr2=0x7fc63806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.559+0000 7fc660400640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc63806b4d0 0x7fc63806d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc648001c80 tx=0x7fc648009040 comp rx=0 tx=0).stop 2024-03-08T06:06:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.559+0000 7fc660400640 1 -- 172.21.15.40:0/1714910369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc658147480 msgr2=0x7fc658143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.559+0000 7fc660400640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc658147480 0x7fc658143bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc64400c500 tx=0x7fc64400c9c0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.559+0000 7fc660400640 1 -- 172.21.15.40:0/1714910369 shutdown_connections 2024-03-08T06:06:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.559+0000 7fc660400640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.559+0000 7fc660400640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc658161840 0x7fc658144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.559+0000 7fc660400640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc658147480 0x7fc658143bd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.559+0000 7fc660400640 1 --2- 172.21.15.40:0/1714910369 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc658146f30 0x7fc6581450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.559+0000 7fc660400640 1 -- 172.21.15.40:0/1714910369 >> 172.21.15.40:0/1714910369 conn(0x7fc6580b16e0 msgr2=0x7fc65806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.560+0000 7fc660400640 1 -- 172.21.15.40:0/1714910369 shutdown_connections 2024-03-08T06:06:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:17.560+0000 7fc660400640 1 -- 172.21.15.40:0/1714910369 wait complete. 2024-03-08T06:06:17.579 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-08T06:06:17.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:17.579 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-08T06:06:17.700 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:06:17.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:17.700 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-08T06:06:17.754 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:06:17.754 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:06:17.815 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054202,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-08T06:06:17.815 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:06:17.815 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:06:17.873 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054202,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-08T06:06:17.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:06:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.101+0000 7fb896000640 1 Processor -- start 2024-03-08T06:06:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.101+0000 7fb896000640 1 -- start start 2024-03-08T06:06:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.102+0000 7fb896000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb890071df0 0x7fb89015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.103+0000 7fb896000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89015ceb0 con 0x7fb890145520 2024-03-08T06:06:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.103+0000 7fb896000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89015c330 con 0x7fb89015bea0 2024-03-08T06:06:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.103+0000 7fb896000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89015c610 con 0x7fb890071df0 2024-03-08T06:06:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.103+0000 7fb894c00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb890145520 0x7fb8900716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41966/0 (socket says 172.21.15.40:41966) 2024-03-08T06:06:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.103+0000 7fb894c00640 1 -- 172.21.15.40:0/3429533271 learned_addr learned my addr 172.21.15.40:0/3429533271 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.103+0000 7fb895600640 1 --2- 172.21.15.40:0/3429533271 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb890071df0 0x7fb89015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.103+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686705978 0 0) 0x7fb89015ceb0 con 0x7fb890145520 2024-03-08T06:06:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb878003520 con 0x7fb890145520 2024-03-08T06:06:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3122431909 0 0) 0x7fb878003520 con 0x7fb890145520 2024-03-08T06:06:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb890071df0 msgr2=0x7fb89015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 --2- 172.21.15.40:0/3429533271 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb890071df0 0x7fb89015d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 >> v1:172.21.15.53:6789/0 conn(0x7fb89015bea0 legacy=0x7fb89006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89015c8f0 con 0x7fb890145520 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fb884001670 con 0x7fb890145520 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 >> v1:172.21.15.40:6789/0 conn(0x7fb890145520 legacy=0x7fb8900716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 --2- 172.21.15.40:0/3429533271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 0x7fb878003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 --2- 172.21.15.40:0/3429533271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb878004800 0x7fb878004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 --2- 172.21.15.40:0/3429533271 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb890071df0 0x7fb8780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89015c610 con 0x7fb8780039e0 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87800baa0 con 0x7fb878004800 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87800bca0 con 0x7fb890071df0 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.104+0000 7fb894c00640 1 --2- 172.21.15.40:0/3429533271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 0x7fb878003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.105+0000 7fb88fe00640 1 --2- 172.21.15.40:0/3429533271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb878004800 0x7fb878004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.105+0000 7fb894c00640 1 -- 172.21.15.40:0/3429533271 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb890071df0 msgr2=0x7fb8780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.105+0000 7fb894c00640 1 --2- 172.21.15.40:0/3429533271 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb890071df0 0x7fb8780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.105+0000 7fb894c00640 1 -- 172.21.15.40:0/3429533271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb878004800 msgr2=0x7fb878004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.105+0000 7fb894c00640 1 --2- 172.21.15.40:0/3429533271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb878004800 0x7fb878004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.105+0000 7fb894c00640 1 -- 172.21.15.40:0/3429533271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878003520 con 0x7fb8780039e0 2024-03-08T06:06:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.106+0000 7fb894c00640 1 --2- 172.21.15.40:0/3429533271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 0x7fb878003e00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb88400c210 tx=0x7fb878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2370e3bb032b2cb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:18.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.106+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884007cf0 con 0x7fb8780039e0 2024-03-08T06:06:18.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.106+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb884030070 con 0x7fb8780039e0 2024-03-08T06:06:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.106+0000 7fb88f400640 1 -- 172.21.15.40:0/3429533271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884005d30 con 0x7fb8780039e0 2024-03-08T06:06:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.107+0000 7fb896000640 1 -- 172.21.15.40:0/3429533271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 msgr2=0x7fb878003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.107+0000 7fb896000640 1 --2- 172.21.15.40:0/3429533271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 0x7fb878003e00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb88400c210 tx=0x7fb878005830 comp rx=0 tx=0).stop 2024-03-08T06:06:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.107+0000 7fb895600640 1 -- 172.21.15.40:0/3429533271 reap_dead start 2024-03-08T06:06:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.107+0000 7fb896000640 1 -- 172.21.15.40:0/3429533271 shutdown_connections 2024-03-08T06:06:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.107+0000 7fb896000640 1 -- 172.21.15.40:0/3429533271 >> 172.21.15.40:0/3429533271 conn(0x7fb8900b1850 msgr2=0x7fb8900b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.107+0000 7fb896000640 1 -- 172.21.15.40:0/3429533271 shutdown_connections 2024-03-08T06:06:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.107+0000 7fb896000640 1 -- 172.21.15.40:0/3429533271 wait complete. 2024-03-08T06:06:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.109+0000 7fb896000640 1 Processor -- start 2024-03-08T06:06:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.109+0000 7fb896000640 1 -- start start 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.110+0000 7fb896000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 0x7fb8901f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.110+0000 7fb896000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8900b41c0 0x7fb890141500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.110+0000 7fb896000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8900b4640 0x7fb8900cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.110+0000 7fb896000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900af1c0 con 0x7fb8780039e0 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.110+0000 7fb896000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900af040 con 0x7fb8900b41c0 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.110+0000 7fb896000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89006e410 con 0x7fb8900b4640 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.110+0000 7fb894c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 0x7fb8901f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.110+0000 7fb894c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 0x7fb8901f38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39716/0 (socket says 172.21.15.40:39716) 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.110+0000 7fb894c00640 1 -- 172.21.15.40:0/4033965082 learned_addr learned my addr 172.21.15.40:0/4033965082 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.111+0000 7fb894c00640 1 -- 172.21.15.40:0/4033965082 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8900b4640 msgr2=0x7fb8900cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.111+0000 7fb894c00640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8900b4640 0x7fb8900cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.111+0000 7fb894c00640 1 -- 172.21.15.40:0/4033965082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8900b41c0 msgr2=0x7fb890141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.111+0000 7fb894c00640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8900b41c0 0x7fb890141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.111+0000 7fb894c00640 1 -- 172.21.15.40:0/4033965082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890141f50 con 0x7fb8780039e0 2024-03-08T06:06:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.111+0000 7fb894c00640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 0x7fb8901f38b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb884001970 tx=0x7fb89006d220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.111+0000 7fb88cc00640 1 -- 172.21.15.40:0/4033965082 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884046070 con 0x7fb8780039e0 2024-03-08T06:06:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.112+0000 7fb88cc00640 1 -- 172.21.15.40:0/4033965082 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb884000ed0 con 0x7fb8780039e0 2024-03-08T06:06:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.112+0000 7fb88cc00640 1 -- 172.21.15.40:0/4033965082 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884001c30 con 0x7fb8780039e0 2024-03-08T06:06:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.112+0000 7fb896000640 1 -- 172.21.15.40:0/4033965082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb890142230 con 0x7fb8780039e0 2024-03-08T06:06:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.112+0000 7fb896000640 1 -- 172.21.15.40:0/4033965082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8901427f0 con 0x7fb8780039e0 2024-03-08T06:06:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.113+0000 7fb896000640 1 -- 172.21.15.40:0/4033965082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb890075100 con 0x7fb8780039e0 2024-03-08T06:06:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.114+0000 7fb88cc00640 1 -- 172.21.15.40:0/4033965082 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb884007070 con 0x7fb8780039e0 2024-03-08T06:06:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.115+0000 7fb88cc00640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb87006b4d0 0x7fb87006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.115+0000 7fb88cc00640 1 -- 172.21.15.40:0/4033965082 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8840ab340 con 0x7fb8780039e0 2024-03-08T06:06:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.115+0000 7fb88fe00640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb87006b4d0 0x7fb87006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.116+0000 7fb88fe00640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb87006b4d0 0x7fb87006d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb87c007fd0 tx=0x7fb87c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:18.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.119+0000 7fb88cc00640 1 -- 172.21.15.40:0/4033965082 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb88407a230 con 0x7fb8780039e0 2024-03-08T06:06:18.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.365+0000 7fb896000640 1 -- 172.21.15.40:0/4033965082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8900cddd0 con 0x7fb8780039e0 2024-03-08T06:06:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.366+0000 7fb88cc00640 1 -- 172.21.15.40:0/4033965082 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4935 (secure 0 0 0) 0x7fb884079b30 con 0x7fb8780039e0 2024-03-08T06:06:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:18.366 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/2194146146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6835","nonce":2194146146},{"type":"v1","addr":"172.21.15.40:6837","nonce":2194146146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/831964898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":831964898},{"type":"v1","addr":"172.21.15.53:6835","nonce":831964898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4723,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2951334634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2951334634},{"type":"v1","addr":"172.21.15.53:6833","nonce":2951334634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:05:11.446139+0000","modified":"2024-03-08T06:05:14.485839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6836/1742457959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1742457959},{"type":"v1","addr":"172.21.15.40:6836","nonce":1742457959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:06:18.367 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2024-03-08T06:06:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.368+0000 7fb896000640 1 -- 172.21.15.40:0/4033965082 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb87006b4d0 msgr2=0x7fb87006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.368+0000 7fb896000640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb87006b4d0 0x7fb87006d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb87c007fd0 tx=0x7fb87c009040 comp rx=0 tx=0).stop 2024-03-08T06:06:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.368+0000 7fb896000640 1 -- 172.21.15.40:0/4033965082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 msgr2=0x7fb8901f38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.368+0000 7fb896000640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 0x7fb8901f38b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb884001970 tx=0x7fb89006d220 comp rx=0 tx=0).stop 2024-03-08T06:06:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.369+0000 7fb896000640 1 -- 172.21.15.40:0/4033965082 shutdown_connections 2024-03-08T06:06:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.369+0000 7fb896000640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb87006b4d0 0x7fb87006d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.369+0000 7fb896000640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8900b4640 0x7fb8900cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.369+0000 7fb896000640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8900b41c0 0x7fb890141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.369+0000 7fb896000640 1 --2- 172.21.15.40:0/4033965082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8780039e0 0x7fb8901f38b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.369+0000 7fb896000640 1 -- 172.21.15.40:0/4033965082 >> 172.21.15.40:0/4033965082 conn(0x7fb8900b1850 msgr2=0x7fb8900adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.369+0000 7fb896000640 1 -- 172.21.15.40:0/4033965082 shutdown_connections 2024-03-08T06:06:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.369+0000 7fb896000640 1 -- 172.21.15.40:0/4033965082 wait complete. 2024-03-08T06:06:18.389 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:06:18.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.608+0000 7ff503e00640 1 Processor -- start 2024-03-08T06:06:18.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.608+0000 7ff503e00640 1 -- start start 2024-03-08T06:06:18.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.610+0000 7ff503e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff50415a770 0x7ff50415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.610+0000 7ff503e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff50406ea40 0x7ff50415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.610+0000 7ff503e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5040cd020 con 0x7ff5040cc310 2024-03-08T06:06:18.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.610+0000 7ff503e00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5040ccdd0 con 0x7ff50406ea40 2024-03-08T06:06:18.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.610+0000 7ff503e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50415e8f0 con 0x7ff50415a770 2024-03-08T06:06:18.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.610+0000 7ff503400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff50406ea40 0x7ff50415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:18.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.610+0000 7ff503400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff50406ea40 0x7ff50415ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39718/0 (socket says 172.21.15.40:39718) 2024-03-08T06:06:18.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.610+0000 7ff503400640 1 -- 172.21.15.40:0/254106158 learned_addr learned my addr 172.21.15.40:0/254106158 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:18.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.611+0000 7ff502a00640 1 --2- 172.21.15.40:0/254106158 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff50415a770 0x7ff50415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:18.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.611+0000 7ff503400640 1 -- 172.21.15.40:0/254106158 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff50415a770 msgr2=0x7ff50415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:18.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.611+0000 7ff503400640 1 --2- 172.21.15.40:0/254106158 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff50415a770 0x7ff50415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.611+0000 7ff503400640 1 -- 172.21.15.40:0/254106158 >> v1:172.21.15.40:6789/0 conn(0x7ff5040cc310 legacy=0x7ff50406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.611+0000 7ff503400640 1 -- 172.21.15.40:0/254106158 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5040cd300 con 0x7ff50406ea40 2024-03-08T06:06:18.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.612+0000 7ff503400640 1 --2- 172.21.15.40:0/254106158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff50406ea40 0x7ff50415ab50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009e50 tx=0x7ff4fc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5894aa95ae4572d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:18.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.612+0000 7ff502000640 1 -- 172.21.15.40:0/254106158 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc02cb00 con 0x7ff50406ea40 2024-03-08T06:06:18.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.612+0000 7ff502000640 1 -- 172.21.15.40:0/254106158 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc02cca0 con 0x7ff50406ea40 2024-03-08T06:06:18.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.612+0000 7ff502000640 1 -- 172.21.15.40:0/254106158 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc034610 con 0x7ff50406ea40 2024-03-08T06:06:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.613+0000 7ff503e00640 1 -- 172.21.15.40:0/254106158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff50406ea40 msgr2=0x7ff50415ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.613+0000 7ff503e00640 1 --2- 172.21.15.40:0/254106158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff50406ea40 0x7ff50415ab50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009e50 tx=0x7ff4fc02bf60 comp rx=0 tx=0).stop 2024-03-08T06:06:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.613+0000 7ff503e00640 1 -- 172.21.15.40:0/254106158 shutdown_connections 2024-03-08T06:06:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.613+0000 7ff503e00640 1 --2- 172.21.15.40:0/254106158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff50406ea40 0x7ff50415ab50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.613+0000 7ff503e00640 1 --2- 172.21.15.40:0/254106158 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff50415a770 0x7ff50415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.613+0000 7ff503e00640 1 -- 172.21.15.40:0/254106158 >> 172.21.15.40:0/254106158 conn(0x7ff5040b1850 msgr2=0x7ff5040b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.613+0000 7ff503e00640 1 -- 172.21.15.40:0/254106158 shutdown_connections 2024-03-08T06:06:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.613+0000 7ff503e00640 1 -- 172.21.15.40:0/254106158 wait complete. 2024-03-08T06:06:18.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.615+0000 7ff503e00640 1 Processor -- start 2024-03-08T06:06:18.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.615+0000 7ff503e00640 1 -- start start 2024-03-08T06:06:18.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.616+0000 7ff503e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff50406ea40 0x7ff504147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.616+0000 7ff503e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5040cc310 0x7ff50414a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.616+0000 7ff503e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff50415a770 0x7ff5041474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.616+0000 7ff503e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040b4e00 con 0x7ff50415a770 2024-03-08T06:06:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.616+0000 7ff503e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50415f2c0 con 0x7ff5040cc310 2024-03-08T06:06:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.616+0000 7ff503e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040cc790 con 0x7ff50406ea40 2024-03-08T06:06:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.616+0000 7ff503400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff50415a770 0x7ff5041474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.616+0000 7ff503400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff50415a770 0x7ff5041474d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39724/0 (socket says 172.21.15.40:39724) 2024-03-08T06:06:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.616+0000 7ff503400640 1 -- 172.21.15.40:0/1892160945 learned_addr learned my addr 172.21.15.40:0/1892160945 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.617+0000 7ff502a00640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff50406ea40 0x7ff504147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.617+0000 7ff4fbe00640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5040cc310 0x7ff50414a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.617+0000 7ff503400640 1 -- 172.21.15.40:0/1892160945 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff50406ea40 msgr2=0x7ff504147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.617+0000 7ff503400640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff50406ea40 0x7ff504147fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.617+0000 7ff503400640 1 -- 172.21.15.40:0/1892160945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5040cc310 msgr2=0x7ff50414a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.617+0000 7ff503400640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5040cc310 0x7ff50414a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.617+0000 7ff503400640 1 -- 172.21.15.40:0/1892160945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff504152ef0 con 0x7ff50415a770 2024-03-08T06:06:18.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.617+0000 7ff503400640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff50415a770 0x7ff5041474d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009f70 tx=0x7ff4fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:18.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.617+0000 7ff4fb400640 1 -- 172.21.15.40:0/1892160945 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc043020 con 0x7ff50415a770 2024-03-08T06:06:18.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.617+0000 7ff4fb400640 1 -- 172.21.15.40:0/1892160945 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc02ce50 con 0x7ff50415a770 2024-03-08T06:06:18.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.617+0000 7ff4fb400640 1 -- 172.21.15.40:0/1892160945 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc009510 con 0x7ff50415a770 2024-03-08T06:06:18.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.618+0000 7ff503e00640 1 -- 172.21.15.40:0/1892160945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5041531d0 con 0x7ff50415a770 2024-03-08T06:06:18.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.618+0000 7ff503e00640 1 -- 172.21.15.40:0/1892160945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff504153890 con 0x7ff50415a770 2024-03-08T06:06:18.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.619+0000 7ff503e00640 1 -- 172.21.15.40:0/1892160945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff504075100 con 0x7ff50415a770 2024-03-08T06:06:18.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.620+0000 7ff4fb400640 1 -- 172.21.15.40:0/1892160945 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4fc046020 con 0x7ff50415a770 2024-03-08T06:06:18.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.621+0000 7ff4fb400640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff4e006b500 0x7ff4e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:18.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.621+0000 7ff4fb400640 1 -- 172.21.15.40:0/1892160945 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4fc0ac720 con 0x7ff50415a770 2024-03-08T06:06:18.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.621+0000 7ff502a00640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff4e006b500 0x7ff4e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:18.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.622+0000 7ff502a00640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff4e006b500 0x7ff4e006d9c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4006fd0 tx=0x7ff4f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:18.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.625+0000 7ff4fb400640 1 -- 172.21.15.40:0/1892160945 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4fc04d030 con 0x7ff50415a770 2024-03-08T06:06:18.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.826+0000 7ff503e00640 1 -- 172.21.15.40:0/1892160945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5040cf680 con 0x7ff50415a770 2024-03-08T06:06:18.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.830+0000 7ff4fb400640 1 -- 172.21.15.40:0/1892160945 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16585 (secure 0 0 0) 0x7ff4fc07b630 con 0x7ff50415a770 2024-03-08T06:06:18.831 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:18.831 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:05:12.978712+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:05:09.544904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T06:05:10.468274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:06:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.832+0000 7ff503e00640 1 -- 172.21.15.40:0/1892160945 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff4e006b500 msgr2=0x7ff4e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.832+0000 7ff503e00640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff4e006b500 0x7ff4e006d9c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4006fd0 tx=0x7ff4f4008040 comp rx=0 tx=0).stop 2024-03-08T06:06:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.832+0000 7ff503e00640 1 -- 172.21.15.40:0/1892160945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff50415a770 msgr2=0x7ff5041474d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.832+0000 7ff503e00640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff50415a770 0x7ff5041474d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009f70 tx=0x7ff4fc004a40 comp rx=0 tx=0).stop 2024-03-08T06:06:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.832+0000 7ff503e00640 1 -- 172.21.15.40:0/1892160945 shutdown_connections 2024-03-08T06:06:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.832+0000 7ff503e00640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff4e006b500 0x7ff4e006d9c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.832+0000 7ff503e00640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff50415a770 0x7ff5041474d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.832+0000 7ff503e00640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5040cc310 0x7ff50414a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.832+0000 7ff503e00640 1 --2- 172.21.15.40:0/1892160945 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff50406ea40 0x7ff504147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:18.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.832+0000 7ff503e00640 1 -- 172.21.15.40:0/1892160945 >> 172.21.15.40:0/1892160945 conn(0x7ff5040b1850 msgr2=0x7ff50406d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:18.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.833+0000 7ff503e00640 1 -- 172.21.15.40:0/1892160945 shutdown_connections 2024-03-08T06:06:18.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:18.833+0000 7ff503e00640 1 -- 172.21.15.40:0/1892160945 wait complete. 2024-03-08T06:06:18.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T06:03:55.558230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T06:05:09.544904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T06:05:10.468274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T06:06:18.852 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T06:06:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.075+0000 7fb47de00640 1 Processor -- start 2024-03-08T06:06:19.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.075+0000 7fb47de00640 1 -- start start 2024-03-08T06:06:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.077+0000 7fb47de00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb4780b3260 0x7fb47806d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.077+0000 7fb47de00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4780ae650 con 0x7fb478072760 2024-03-08T06:06:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.077+0000 7fb47de00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4780aee60 con 0x7fb478071a60 2024-03-08T06:06:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.077+0000 7fb47de00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780ad460 con 0x7fb4780b3260 2024-03-08T06:06:19.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.077+0000 7fb46e000640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb478072760 0x7fb4780b2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41990/0 (socket says 172.21.15.40:41990) 2024-03-08T06:06:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.077+0000 7fb46e000640 1 -- 172.21.15.40:0/139119177 learned_addr learned my addr 172.21.15.40:0/139119177 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb477400640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb4780b3260 0x7fb47806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169842198 0 0) 0x7fb4780ae650 con 0x7fb478072760 2024-03-08T06:06:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb45c003520 con 0x7fb478072760 2024-03-08T06:06:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3138349229 0 0) 0x7fb45c003520 con 0x7fb478072760 2024-03-08T06:06:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb4780b3260 msgr2=0x7fb47806d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb4780b3260 0x7fb47806d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 >> v1:172.21.15.53:6789/0 conn(0x7fb478071a60 legacy=0x7fb4780adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780ad740 con 0x7fb478072760 2024-03-08T06:06:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fb460001670 con 0x7fb478072760 2024-03-08T06:06:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 >> v1:172.21.15.40:6789/0 conn(0x7fb478072760 legacy=0x7fb4780b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 0x7fb45c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb4780b3260 0x7fb45c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb45c004d40 0x7fb45c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780ad460 con 0x7fb45c0039e0 2024-03-08T06:06:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c00baf0 con 0x7fb45c004d40 2024-03-08T06:06:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.078+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c00bcf0 con 0x7fb4780b3260 2024-03-08T06:06:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.079+0000 7fb476a00640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 0x7fb45c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.079+0000 7fb477400640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb45c004d40 0x7fb45c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.079+0000 7fb46e000640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb4780b3260 0x7fb45c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.079+0000 7fb476a00640 1 -- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb4780b3260 msgr2=0x7fb45c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:19.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.079+0000 7fb476a00640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb4780b3260 0x7fb45c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.079+0000 7fb476a00640 1 -- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb45c004d40 msgr2=0x7fb45c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.079+0000 7fb476a00640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb45c004d40 0x7fb45c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.079+0000 7fb476a00640 1 -- 172.21.15.40:0/139119177 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c003520 con 0x7fb45c0039e0 2024-03-08T06:06:19.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.079+0000 7fb46e000640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb4780b3260 0x7fb45c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.080+0000 7fb476a00640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 0x7fb45c003e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb468001e50 tx=0x7fb45c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810b1dff7d92ba44 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.081+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468001a70 con 0x7fb45c0039e0 2024-03-08T06:06:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.081+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468003b70 con 0x7fb45c0039e0 2024-03-08T06:06:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.081+0000 7fb476000640 1 -- 172.21.15.40:0/139119177 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4680050d0 con 0x7fb45c0039e0 2024-03-08T06:06:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.081+0000 7fb47de00640 1 -- 172.21.15.40:0/139119177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 msgr2=0x7fb45c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.081+0000 7fb47de00640 1 --2- 172.21.15.40:0/139119177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 0x7fb45c003e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb468001e50 tx=0x7fb45c005830 comp rx=0 tx=0).stop 2024-03-08T06:06:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.081+0000 7fb477400640 1 -- 172.21.15.40:0/139119177 reap_dead start 2024-03-08T06:06:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.081+0000 7fb47de00640 1 -- 172.21.15.40:0/139119177 shutdown_connections 2024-03-08T06:06:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.081+0000 7fb47de00640 1 -- 172.21.15.40:0/139119177 >> 172.21.15.40:0/139119177 conn(0x7fb4780cfaa0 msgr2=0x7fb4780cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.081+0000 7fb47de00640 1 -- 172.21.15.40:0/139119177 shutdown_connections 2024-03-08T06:06:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.081+0000 7fb47de00640 1 -- 172.21.15.40:0/139119177 wait complete. 2024-03-08T06:06:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.083+0000 7fb47de00640 1 Processor -- start 2024-03-08T06:06:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.083+0000 7fb47de00640 1 -- start start 2024-03-08T06:06:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.084+0000 7fb47de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 0x7fb47815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.084+0000 7fb47de00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb47814c790 0x7fb4781f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.084+0000 7fb47de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4781e7ef0 0x7fb4781e8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.084+0000 7fb47de00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780ad110 con 0x7fb45c0039e0 2024-03-08T06:06:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.084+0000 7fb47de00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780aeb10 con 0x7fb4781e7ef0 2024-03-08T06:06:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.084+0000 7fb47de00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb478071e40 con 0x7fb47814c790 2024-03-08T06:06:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.085+0000 7fb476a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 0x7fb47815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.085+0000 7fb476a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 0x7fb47815ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39742/0 (socket says 172.21.15.40:39742) 2024-03-08T06:06:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.085+0000 7fb476a00640 1 -- 172.21.15.40:0/3876181216 learned_addr learned my addr 172.21.15.40:0/3876181216 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.085+0000 7fb46e000640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb47814c790 0x7fb4781f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.085+0000 7fb477400640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4781e7ef0 0x7fb4781e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.085+0000 7fb476a00640 1 -- 172.21.15.40:0/3876181216 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb47814c790 msgr2=0x7fb4781f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.085+0000 7fb476a00640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb47814c790 0x7fb4781f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.085+0000 7fb476a00640 1 -- 172.21.15.40:0/3876181216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4781e7ef0 msgr2=0x7fb4781e8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.085+0000 7fb476a00640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4781e7ef0 0x7fb4781e8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.085+0000 7fb476a00640 1 -- 172.21.15.40:0/3876181216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47815a090 con 0x7fb45c0039e0 2024-03-08T06:06:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.085+0000 7fb476a00640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 0x7fb47815ce60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb46802bfb0 tx=0x7fb4680014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.086+0000 7fb46fe00640 1 -- 172.21.15.40:0/3876181216 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468007db0 con 0x7fb45c0039e0 2024-03-08T06:06:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.086+0000 7fb46fe00640 1 -- 172.21.15.40:0/3876181216 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46803f030 con 0x7fb45c0039e0 2024-03-08T06:06:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.086+0000 7fb46fe00640 1 -- 172.21.15.40:0/3876181216 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468007030 con 0x7fb45c0039e0 2024-03-08T06:06:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.086+0000 7fb47de00640 1 -- 172.21.15.40:0/3876181216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47815a370 con 0x7fb45c0039e0 2024-03-08T06:06:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.086+0000 7fb47de00640 1 -- 172.21.15.40:0/3876181216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47815a930 con 0x7fb45c0039e0 2024-03-08T06:06:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.087+0000 7fb47de00640 1 -- 172.21.15.40:0/3876181216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb478067cb0 con 0x7fb45c0039e0 2024-03-08T06:06:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.088+0000 7fb46fe00640 1 -- 172.21.15.40:0/3876181216 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4680071d0 con 0x7fb45c0039e0 2024-03-08T06:06:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.089+0000 7fb46fe00640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb45406b4d0 0x7fb45406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.089+0000 7fb46fe00640 1 -- 172.21.15.40:0/3876181216 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4680ac5e0 con 0x7fb45c0039e0 2024-03-08T06:06:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.089+0000 7fb46e000640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb45406b4d0 0x7fb45406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.090+0000 7fb46e000640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb45406b4d0 0x7fb45406d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb460001200 tx=0x7fb460009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.093+0000 7fb46fe00640 1 -- 172.21.15.40:0/3876181216 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb46807b4f0 con 0x7fb45c0039e0 2024-03-08T06:06:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.290+0000 7fb47de00640 1 -- 172.21.15.40:0/3876181216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4780724d0 con 0x7fb45c0039e0 2024-03-08T06:06:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.291+0000 7fb46fe00640 1 -- 172.21.15.40:0/3876181216 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb46807adf0 con 0x7fb45c0039e0 2024-03-08T06:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-03-08T06:06:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.293+0000 7fb47de00640 1 -- 172.21.15.40:0/3876181216 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb45406b4d0 msgr2=0x7fb45406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.293+0000 7fb47de00640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb45406b4d0 0x7fb45406d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb460001200 tx=0x7fb460009040 comp rx=0 tx=0).stop 2024-03-08T06:06:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.293+0000 7fb47de00640 1 -- 172.21.15.40:0/3876181216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 msgr2=0x7fb47815ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.293+0000 7fb47de00640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 0x7fb47815ce60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb46802bfb0 tx=0x7fb4680014a0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.294+0000 7fb47de00640 1 -- 172.21.15.40:0/3876181216 shutdown_connections 2024-03-08T06:06:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.294+0000 7fb47de00640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb45406b4d0 0x7fb45406d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.294+0000 7fb47de00640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4781e7ef0 0x7fb4781e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.294+0000 7fb47de00640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb47814c790 0x7fb4781f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.294+0000 7fb47de00640 1 --2- 172.21.15.40:0/3876181216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb45c0039e0 0x7fb47815ce60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.294+0000 7fb47de00640 1 -- 172.21.15.40:0/3876181216 >> 172.21.15.40:0/3876181216 conn(0x7fb4780cfaa0 msgr2=0x7fb4780b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.294+0000 7fb47de00640 1 -- 172.21.15.40:0/3876181216 shutdown_connections 2024-03-08T06:06:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.294+0000 7fb47de00640 1 -- 172.21.15.40:0/3876181216 wait complete. 2024-03-08T06:06:19.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T06:06:19.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.542+0000 7f65e1c00640 1 Processor -- start 2024-03-08T06:06:19.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.542+0000 7f65e1c00640 1 -- start start 2024-03-08T06:06:19.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.544+0000 7f65e1c00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65dc1584a0 0x7f65dc071950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65e1c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65dc071e90 0x7f65dc15e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65e1c00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65dc15f390 con 0x7f65dc15eb50 2024-03-08T06:06:19.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65e1c00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65dc06d820 con 0x7f65dc1584a0 2024-03-08T06:06:19.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65e1c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc06db00 con 0x7f65dc071e90 2024-03-08T06:06:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65daa00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65dc1584a0 0x7f65dc071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65daa00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65dc1584a0 0x7f65dc071950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39750/0 (socket says 172.21.15.40:39750) 2024-03-08T06:06:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65daa00640 1 -- 172.21.15.40:0/1818015801 learned_addr learned my addr 172.21.15.40:0/1818015801 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65da000640 1 --2- 172.21.15.40:0/1818015801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65dc071e90 0x7f65dc15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65daa00640 1 -- 172.21.15.40:0/1818015801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65dc071e90 msgr2=0x7f65dc15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65daa00640 1 --2- 172.21.15.40:0/1818015801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65dc071e90 0x7f65dc15e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65daa00640 1 -- 172.21.15.40:0/1818015801 >> v1:172.21.15.53:6789/0 conn(0x7f65dc15eb50 legacy=0x7f65dc161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65daa00640 1 -- 172.21.15.40:0/1818015801 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc06dde0 con 0x7f65dc1584a0 2024-03-08T06:06:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.545+0000 7f65da000640 1 --2- 172.21.15.40:0/1818015801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65dc071e90 0x7f65dc15e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:19.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.547+0000 7f65daa00640 1 --2- 172.21.15.40:0/1818015801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65dc1584a0 0x7f65dc071950 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009870 tx=0x7f65cc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e7dc4292bef9c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:19.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.547+0000 7f65d9600640 1 -- 172.21.15.40:0/1818015801 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0317c0 con 0x7f65dc1584a0 2024-03-08T06:06:19.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.547+0000 7f65d9600640 1 -- 172.21.15.40:0/1818015801 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc037070 con 0x7f65dc1584a0 2024-03-08T06:06:19.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.547+0000 7f65d9600640 1 -- 172.21.15.40:0/1818015801 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc031b80 con 0x7f65dc1584a0 2024-03-08T06:06:19.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.548+0000 7f65e1c00640 1 -- 172.21.15.40:0/1818015801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65dc1584a0 msgr2=0x7f65dc071950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.548+0000 7f65e1c00640 1 --2- 172.21.15.40:0/1818015801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65dc1584a0 0x7f65dc071950 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009870 tx=0x7f65cc02b860 comp rx=0 tx=0).stop 2024-03-08T06:06:19.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.548+0000 7f65e1c00640 1 -- 172.21.15.40:0/1818015801 shutdown_connections 2024-03-08T06:06:19.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.548+0000 7f65e1c00640 1 --2- 172.21.15.40:0/1818015801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65dc071e90 0x7f65dc15e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.548+0000 7f65e1c00640 1 --2- 172.21.15.40:0/1818015801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65dc1584a0 0x7f65dc071950 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.548+0000 7f65e1c00640 1 -- 172.21.15.40:0/1818015801 >> 172.21.15.40:0/1818015801 conn(0x7f65dc06ae30 msgr2=0x7f65dc06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:19.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.548+0000 7f65e1c00640 1 -- 172.21.15.40:0/1818015801 shutdown_connections 2024-03-08T06:06:19.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.548+0000 7f65e1c00640 1 -- 172.21.15.40:0/1818015801 wait complete. 2024-03-08T06:06:19.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.550+0000 7f65e1c00640 1 Processor -- start 2024-03-08T06:06:19.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.550+0000 7f65e1c00640 1 -- start start 2024-03-08T06:06:19.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65e1c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc071e90 0x7f65dc0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65e1c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65dc1584a0 0x7f65dc160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65e1c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65dc15eb50 0x7f65dc161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65e1c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc06e5e0 con 0x7f65dc1584a0 2024-03-08T06:06:19.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65e1c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc158880 con 0x7f65dc071e90 2024-03-08T06:06:19.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65e1c00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1f4450 con 0x7f65dc15eb50 2024-03-08T06:06:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65da000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65dc1584a0 0x7f65dc160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65da000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65dc1584a0 0x7f65dc160af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39756/0 (socket says 172.21.15.40:39756) 2024-03-08T06:06:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65da000640 1 -- 172.21.15.40:0/1152413772 learned_addr learned my addr 172.21.15.40:0/1152413772 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65daa00640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc071e90 0x7f65dc0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65db400640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65dc15eb50 0x7f65dc161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.551+0000 7f65da000640 1 -- 172.21.15.40:0/1152413772 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65dc15eb50 msgr2=0x7f65dc161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.552+0000 7f65da000640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65dc15eb50 0x7f65dc161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.552+0000 7f65da000640 1 -- 172.21.15.40:0/1152413772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc071e90 msgr2=0x7f65dc0b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.552+0000 7f65da000640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc071e90 0x7f65dc0b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.552+0000 7f65da000640 1 -- 172.21.15.40:0/1152413772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc1f4730 con 0x7f65dc1584a0 2024-03-08T06:06:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.552+0000 7f65da000640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65dc1584a0 0x7f65dc160af0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f65c400ce30 tx=0x7f65c400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:19.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.552+0000 7f65d2a00640 1 -- 172.21.15.40:0/1152413772 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c400f890 con 0x7f65dc1584a0 2024-03-08T06:06:19.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.552+0000 7f65d2a00640 1 -- 172.21.15.40:0/1152413772 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c400fa30 con 0x7f65dc1584a0 2024-03-08T06:06:19.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.552+0000 7f65d2a00640 1 -- 172.21.15.40:0/1152413772 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c400fe00 con 0x7f65dc1584a0 2024-03-08T06:06:19.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.552+0000 7f65e1c00640 1 -- 172.21.15.40:0/1152413772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc0b0c60 con 0x7f65dc1584a0 2024-03-08T06:06:19.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.553+0000 7f65e1c00640 1 -- 172.21.15.40:0/1152413772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc0b1220 con 0x7f65dc1584a0 2024-03-08T06:06:19.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.554+0000 7f65e1c00640 1 -- 172.21.15.40:0/1152413772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65dc067cb0 con 0x7f65dc1584a0 2024-03-08T06:06:19.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.555+0000 7f65d2a00640 1 -- 172.21.15.40:0/1152413772 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65c4018400 con 0x7f65dc1584a0 2024-03-08T06:06:19.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.556+0000 7f65d2a00640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f65b806b4d0 0x7f65b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:19.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.556+0000 7f65d2a00640 1 -- 172.21.15.40:0/1152413772 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65c408afa0 con 0x7f65dc1584a0 2024-03-08T06:06:19.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.556+0000 7f65daa00640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f65b806b4d0 0x7f65b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:19.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.557+0000 7f65daa00640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f65b806b4d0 0x7f65b806d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f65cc036000 tx=0x7f65cc03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:19.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.561+0000 7f65d2a00640 1 -- 172.21.15.40:0/1152413772 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65c4014030 con 0x7f65dc1584a0 2024-03-08T06:06:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.761+0000 7f65e1c00640 1 -- 172.21.15.40:0/1152413772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65dc0b0a40 con 0x7f65dc1584a0 2024-03-08T06:06:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.762+0000 7f65d2a00640 1 -- 172.21.15.40:0/1152413772 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65c400ae20 con 0x7f65dc1584a0 2024-03-08T06:06:19.763 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-03-08T06:06:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 -- 172.21.15.40:0/1152413772 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f65b806b4d0 msgr2=0x7f65b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f65b806b4d0 0x7f65b806d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f65cc036000 tx=0x7f65cc03d040 comp rx=0 tx=0).stop 2024-03-08T06:06:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 -- 172.21.15.40:0/1152413772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65dc1584a0 msgr2=0x7f65dc160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65dc1584a0 0x7f65dc160af0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f65c400ce30 tx=0x7f65c400a650 comp rx=0 tx=0).stop 2024-03-08T06:06:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 -- 172.21.15.40:0/1152413772 shutdown_connections 2024-03-08T06:06:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f65b806b4d0 0x7f65b806d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65dc15eb50 0x7f65dc161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65dc1584a0 0x7f65dc160af0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 --2- 172.21.15.40:0/1152413772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc071e90 0x7f65dc0b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 -- 172.21.15.40:0/1152413772 >> 172.21.15.40:0/1152413772 conn(0x7f65dc06ae30 msgr2=0x7f65dc15f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 -- 172.21.15.40:0/1152413772 shutdown_connections 2024-03-08T06:06:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:19.765+0000 7f65e1c00640 1 -- 172.21.15.40:0/1152413772 wait complete. 2024-03-08T06:06:19.784 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T06:06:20.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.011+0000 7f2ddf000640 1 Processor -- start 2024-03-08T06:06:20.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.011+0000 7f2ddf000640 1 -- start start 2024-03-08T06:06:20.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.012+0000 7f2ddf000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2dd8145590 0x7f2dd80cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.012+0000 7f2ddf000640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2dd80ccc70 0x7f2dd80cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.012+0000 7f2ddf000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd8071990 con 0x7f2dd815f7f0 2024-03-08T06:06:20.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.012+0000 7f2ddf000640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd80cd5f0 con 0x7f2dd80ccc70 2024-03-08T06:06:20.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.012+0000 7f2ddf000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd8071c70 con 0x7f2dd8145590 2024-03-08T06:06:20.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.013+0000 7f2ddd000640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2dd815f7f0 0x7f2dd815fc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42004/0 (socket says 172.21.15.40:42004) 2024-03-08T06:06:20.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.013+0000 7f2ddd000640 1 -- 172.21.15.40:0/1796003373 learned_addr learned my addr 172.21.15.40:0/1796003373 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:20.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.013+0000 7f2dd7e00640 1 --2- 172.21.15.40:0/1796003373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2dd8145590 0x7f2dd80cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.013+0000 7f2dd7400640 1 --2- 172.21.15.40:0/1796003373 >> v2:172.21.15.53:3300/0 conn(0x7f2dd80ccc70 0x7f2dd80cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.013+0000 7f2dd6a00640 1 -- 172.21.15.40:0/1796003373 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494210345 0 0) 0x7f2dd8071990 con 0x7f2dd815f7f0 2024-03-08T06:06:20.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.013+0000 7f2dd6a00640 1 -- 172.21.15.40:0/1796003373 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dbc003520 con 0x7f2dd815f7f0 2024-03-08T06:06:20.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.014+0000 7f2dd7e00640 1 -- 172.21.15.40:0/1796003373 >> v2:172.21.15.53:3300/0 conn(0x7f2dd80ccc70 msgr2=0x7f2dd80cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:20.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.014+0000 7f2dd7e00640 1 --2- 172.21.15.40:0/1796003373 >> v2:172.21.15.53:3300/0 conn(0x7f2dd80ccc70 0x7f2dd80cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.014+0000 7f2dd7e00640 1 -- 172.21.15.40:0/1796003373 >> v1:172.21.15.40:6789/0 conn(0x7f2dd815f7f0 legacy=0x7f2dd815fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.014+0000 7f2dd7e00640 1 -- 172.21.15.40:0/1796003373 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd8071f50 con 0x7f2dd8145590 2024-03-08T06:06:20.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.015+0000 7f2dd7e00640 1 --2- 172.21.15.40:0/1796003373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2dd8145590 0x7f2dd80cf1a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009d90 tx=0x7f2dc802beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8afa2e823248948 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:20.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.016+0000 7f2dd6a00640 1 -- 172.21.15.40:0/1796003373 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8039070 con 0x7f2dd8145590 2024-03-08T06:06:20.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.016+0000 7f2dd6a00640 1 -- 172.21.15.40:0/1796003373 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc802cc60 con 0x7f2dd8145590 2024-03-08T06:06:20.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.016+0000 7f2dd6a00640 1 -- 172.21.15.40:0/1796003373 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8034640 con 0x7f2dd8145590 2024-03-08T06:06:20.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.016+0000 7f2ddf000640 1 -- 172.21.15.40:0/1796003373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2dd8145590 msgr2=0x7f2dd80cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.016+0000 7f2ddf000640 1 --2- 172.21.15.40:0/1796003373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2dd8145590 0x7f2dd80cf1a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009d90 tx=0x7f2dc802beb0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.016+0000 7f2ddf000640 1 -- 172.21.15.40:0/1796003373 shutdown_connections 2024-03-08T06:06:20.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.016+0000 7f2ddf000640 1 --2- 172.21.15.40:0/1796003373 >> v2:172.21.15.53:3300/0 conn(0x7f2dd80ccc70 0x7f2dd80cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.016+0000 7f2ddf000640 1 --2- 172.21.15.40:0/1796003373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2dd8145590 0x7f2dd80cf1a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.016+0000 7f2ddf000640 1 -- 172.21.15.40:0/1796003373 >> 172.21.15.40:0/1796003373 conn(0x7f2dd80cfaa0 msgr2=0x7f2dd80cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:20.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.017+0000 7f2ddf000640 1 -- 172.21.15.40:0/1796003373 shutdown_connections 2024-03-08T06:06:20.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.017+0000 7f2ddf000640 1 -- 172.21.15.40:0/1796003373 wait complete. 2024-03-08T06:06:20.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.018+0000 7f2ddf000640 1 Processor -- start 2024-03-08T06:06:20.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.018+0000 7f2ddf000640 1 -- start start 2024-03-08T06:06:20.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.019+0000 7f2ddf000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd80ccc70 0x7f2dd8138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.019+0000 7f2ddf000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2dd8145590 0x7f2dd813b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.019+0000 7f2ddf000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd815f7f0 0x7f2dd813bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.019+0000 7f2ddf000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd8145970 con 0x7f2dd80ccc70 2024-03-08T06:06:20.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.019+0000 7f2ddf000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd8072510 con 0x7f2dd815f7f0 2024-03-08T06:06:20.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.019+0000 7f2ddf000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80cd960 con 0x7f2dd8145590 2024-03-08T06:06:20.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.020+0000 7f2dd7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd80ccc70 0x7f2dd8138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.020+0000 7f2dd7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd80ccc70 0x7f2dd8138ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39772/0 (socket says 172.21.15.40:39772) 2024-03-08T06:06:20.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.020+0000 7f2dd7e00640 1 -- 172.21.15.40:0/2163316822 learned_addr learned my addr 172.21.15.40:0/2163316822 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:20.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.020+0000 7f2dd7e00640 1 -- 172.21.15.40:0/2163316822 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2dd8145590 msgr2=0x7f2dd813b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.020+0000 7f2dd7e00640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2dd8145590 0x7f2dd813b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.020+0000 7f2dd7e00640 1 -- 172.21.15.40:0/2163316822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd815f7f0 msgr2=0x7f2dd813bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.020+0000 7f2ddd000640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd815f7f0 0x7f2dd813bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.020+0000 7f2dd7e00640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd815f7f0 0x7f2dd813bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.020+0000 7f2dd7e00640 1 -- 172.21.15.40:0/2163316822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd81f42b0 con 0x7f2dd80ccc70 2024-03-08T06:06:20.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.021+0000 7f2dd7e00640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd80ccc70 0x7f2dd8138ff0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8037040 tx=0x7f2dc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:20.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.021+0000 7f2dcfe00640 1 -- 172.21.15.40:0/2163316822 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8039050 con 0x7f2dd80ccc70 2024-03-08T06:06:20.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.021+0000 7f2dcfe00640 1 -- 172.21.15.40:0/2163316822 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc8045070 con 0x7f2dd80ccc70 2024-03-08T06:06:20.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.021+0000 7f2dcfe00640 1 -- 172.21.15.40:0/2163316822 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8009450 con 0x7f2dd80ccc70 2024-03-08T06:06:20.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.021+0000 7f2ddf000640 1 -- 172.21.15.40:0/2163316822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd813c660 con 0x7f2dd80ccc70 2024-03-08T06:06:20.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.021+0000 7f2ddf000640 1 -- 172.21.15.40:0/2163316822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd813cf00 con 0x7f2dd80ccc70 2024-03-08T06:06:20.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.023+0000 7f2ddf000640 1 -- 172.21.15.40:0/2163316822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd8067cb0 con 0x7f2dd80ccc70 2024-03-08T06:06:20.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.023+0000 7f2dcfe00640 1 -- 172.21.15.40:0/2163316822 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2dc803e850 con 0x7f2dd80ccc70 2024-03-08T06:06:20.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.024+0000 7f2dcfe00640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2db406b4d0 0x7f2db406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.024+0000 7f2dcfe00640 1 -- 172.21.15.40:0/2163316822 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2dc80ace60 con 0x7f2dd80ccc70 2024-03-08T06:06:20.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.025+0000 7f2dd7400640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2db406b4d0 0x7f2db406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.025+0000 7f2dd7400640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2db406b4d0 0x7f2db406d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006a90 tx=0x7f2dc0006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:20.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.029+0000 7f2dcfe00640 1 -- 172.21.15.40:0/2163316822 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2dc8032e20 con 0x7f2dd80ccc70 2024-03-08T06:06:20.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.237+0000 7f2ddf000640 1 -- 172.21.15.40:0/2163316822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2dd80ce8d0 con 0x7f2dd80ccc70 2024-03-08T06:06:20.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.237+0000 7f2dcfe00640 1 -- 172.21.15.40:0/2163316822 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2dc807bcd0 con 0x7f2dd80ccc70 2024-03-08T06:06:20.238 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-03-08T06:06:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.240+0000 7f2ddf000640 1 -- 172.21.15.40:0/2163316822 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2db406b4d0 msgr2=0x7f2db406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.240+0000 7f2ddf000640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2db406b4d0 0x7f2db406d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006a90 tx=0x7f2dc0006a20 comp rx=0 tx=0).stop 2024-03-08T06:06:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.240+0000 7f2ddf000640 1 -- 172.21.15.40:0/2163316822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd80ccc70 msgr2=0x7f2dd8138ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.240+0000 7f2ddf000640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd80ccc70 0x7f2dd8138ff0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8037040 tx=0x7f2dc8004a40 comp rx=0 tx=0).stop 2024-03-08T06:06:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.240+0000 7f2ddf000640 1 -- 172.21.15.40:0/2163316822 shutdown_connections 2024-03-08T06:06:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.240+0000 7f2ddf000640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2db406b4d0 0x7f2db406d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.240+0000 7f2ddf000640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dd815f7f0 0x7f2dd813bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.240+0000 7f2ddf000640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2dd8145590 0x7f2dd813b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.240+0000 7f2ddf000640 1 --2- 172.21.15.40:0/2163316822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd80ccc70 0x7f2dd8138ff0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.240+0000 7f2ddf000640 1 -- 172.21.15.40:0/2163316822 >> 172.21.15.40:0/2163316822 conn(0x7f2dd80cfaa0 msgr2=0x7f2dd80b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.241+0000 7f2ddf000640 1 -- 172.21.15.40:0/2163316822 shutdown_connections 2024-03-08T06:06:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.241+0000 7f2ddf000640 1 -- 172.21.15.40:0/2163316822 wait complete. 2024-03-08T06:06:20.260 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:06:20.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.477+0000 7f9168600640 1 Processor -- start 2024-03-08T06:06:20.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.477+0000 7f9168600640 1 -- start start 2024-03-08T06:06:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.479+0000 7f9168600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9160071ab0 0x7f916015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.479+0000 7f9168600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 0x7f916015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.479+0000 7f9168600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916015d0c0 con 0x7f916006d600 2024-03-08T06:06:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.479+0000 7f9168600640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916006ef40 con 0x7f9160071ab0 2024-03-08T06:06:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.479+0000 7f9168600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916006da80 con 0x7f9160071e90 2024-03-08T06:06:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.480+0000 7f9166600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 0x7f916015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.480+0000 7f9165c00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9160071ab0 0x7f916015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.480+0000 7f9166600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 0x7f916015d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:33934/0 (socket says 172.21.15.40:33934) 2024-03-08T06:06:20.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.480+0000 7f9166600640 1 -- 172.21.15.40:0/789735450 learned_addr learned my addr 172.21.15.40:0/789735450 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:20.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.481+0000 7f9166600640 1 -- 172.21.15.40:0/789735450 >> v2:172.21.15.53:3300/0 conn(0x7f9160071ab0 msgr2=0x7f916015cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:20.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.481+0000 7f9166600640 1 --2- 172.21.15.40:0/789735450 >> v2:172.21.15.53:3300/0 conn(0x7f9160071ab0 0x7f916015cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.481+0000 7f9166600640 1 -- 172.21.15.40:0/789735450 >> v1:172.21.15.53:6789/0 conn(0x7f916006d600 legacy=0x7f9160144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.481+0000 7f9166600640 1 -- 172.21.15.40:0/789735450 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916006dd60 con 0x7f9160071e90 2024-03-08T06:06:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.482+0000 7f9166600640 1 --2- 172.21.15.40:0/789735450 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 0x7f916015d3a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9158009e50 tx=0x7f915802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a1022eeaa8c7d9a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:20.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.483+0000 7f915fe00640 1 -- 172.21.15.40:0/789735450 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915802cb00 con 0x7f9160071e90 2024-03-08T06:06:20.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.483+0000 7f915fe00640 1 -- 172.21.15.40:0/789735450 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915802cca0 con 0x7f9160071e90 2024-03-08T06:06:20.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.483+0000 7f915fe00640 1 -- 172.21.15.40:0/789735450 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158034610 con 0x7f9160071e90 2024-03-08T06:06:20.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.483+0000 7f9168600640 1 -- 172.21.15.40:0/789735450 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 msgr2=0x7f916015d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.483+0000 7f9168600640 1 --2- 172.21.15.40:0/789735450 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 0x7f916015d3a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9158009e50 tx=0x7f915802bf60 comp rx=0 tx=0).stop 2024-03-08T06:06:20.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.484+0000 7f9168600640 1 -- 172.21.15.40:0/789735450 shutdown_connections 2024-03-08T06:06:20.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.484+0000 7f9168600640 1 --2- 172.21.15.40:0/789735450 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 0x7f916015d3a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.484+0000 7f9168600640 1 --2- 172.21.15.40:0/789735450 >> v2:172.21.15.53:3300/0 conn(0x7f9160071ab0 0x7f916015cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.484+0000 7f9168600640 1 -- 172.21.15.40:0/789735450 >> 172.21.15.40:0/789735450 conn(0x7f916006ae30 msgr2=0x7f916006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.484+0000 7f9168600640 1 -- 172.21.15.40:0/789735450 shutdown_connections 2024-03-08T06:06:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.484+0000 7f9168600640 1 -- 172.21.15.40:0/789735450 wait complete. 2024-03-08T06:06:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.485+0000 7f9168600640 1 Processor -- start 2024-03-08T06:06:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.486+0000 7f9168600640 1 -- start start 2024-03-08T06:06:20.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.486+0000 7f9168600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916006d600 0x7f916015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9168600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9160071ab0 0x7f9160143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9168600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 0x7f91600b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9168600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916006e600 con 0x7f9160071ab0 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9168600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916006e320 con 0x7f916006d600 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9168600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9160145470 con 0x7f9160071e90 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9165200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9160071ab0 0x7f9160143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9165200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9160071ab0 0x7f9160143d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39780/0 (socket says 172.21.15.40:39780) 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9165200640 1 -- 172.21.15.40:0/2937892808 learned_addr learned my addr 172.21.15.40:0/2937892808 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9165c00640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916006d600 0x7f916015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9166600640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 0x7f91600b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9165200640 1 -- 172.21.15.40:0/2937892808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 msgr2=0x7f91600b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9165200640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 0x7f91600b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9165200640 1 -- 172.21.15.40:0/2937892808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916006d600 msgr2=0x7f916015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9165200640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916006d600 0x7f916015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.487+0000 7f9165200640 1 -- 172.21.15.40:0/2937892808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9160144250 con 0x7f9160071ab0 2024-03-08T06:06:20.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.488+0000 7f9165200640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9160071ab0 0x7f9160143d10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c3a0 tx=0x7f914c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:20.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.488+0000 7f915d600640 1 -- 172.21.15.40:0/2937892808 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c0052d0 con 0x7f9160071ab0 2024-03-08T06:06:20.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.488+0000 7f915d600640 1 -- 172.21.15.40:0/2937892808 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c005490 con 0x7f9160071ab0 2024-03-08T06:06:20.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.488+0000 7f9168600640 1 -- 172.21.15.40:0/2937892808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91601444d0 con 0x7f9160071ab0 2024-03-08T06:06:20.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.488+0000 7f915d600640 1 -- 172.21.15.40:0/2937892808 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c007260 con 0x7f9160071ab0 2024-03-08T06:06:20.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.488+0000 7f9168600640 1 -- 172.21.15.40:0/2937892808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91600cd5c0 con 0x7f9160071ab0 2024-03-08T06:06:20.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.489+0000 7f9168600640 1 -- 172.21.15.40:0/2937892808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9160067cb0 con 0x7f9160071ab0 2024-03-08T06:06:20.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.490+0000 7f915d600640 1 -- 172.21.15.40:0/2937892808 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f914c007400 con 0x7f9160071ab0 2024-03-08T06:06:20.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.491+0000 7f915d600640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f914006b520 0x7f914006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.491+0000 7f915d600640 1 -- 172.21.15.40:0/2937892808 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f914c089720 con 0x7f9160071ab0 2024-03-08T06:06:20.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.492+0000 7f9165c00640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f914006b520 0x7f914006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.492+0000 7f9165c00640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f914006b520 0x7f914006d9e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9154009ad0 tx=0x7f9154008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:20.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.496+0000 7f915d600640 1 -- 172.21.15.40:0/2937892808 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9160144250 con 0x7f9160071ab0 2024-03-08T06:06:20.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.738+0000 7f9168600640 1 -- 172.21.15.40:0/2937892808 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f916015efd0 con 0x7f9160071ab0 2024-03-08T06:06:20.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.740+0000 7f915d600640 1 -- 172.21.15.40:0/2937892808 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f916015d0c0 con 0x7f9160071ab0 2024-03-08T06:06:20.740 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:20.740 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/2194146146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6835","nonce":2194146146},{"type":"v1","addr":"172.21.15.40:6837","nonce":2194146146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/831964898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":831964898},{"type":"v1","addr":"172.21.15.53:6835","nonce":831964898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4723,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2951334634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2951334634},{"type":"v1","addr":"172.21.15.53:6833","nonce":2951334634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:05:11.446139+0000","modified":"2024-03-08T06:05:14.485839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6836/1742457959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1742457959},{"type":"v1","addr":"172.21.15.40:6836","nonce":1742457959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:06:20.741 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2024-03-08T06:06:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.742+0000 7f9168600640 1 -- 172.21.15.40:0/2937892808 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f914006b520 msgr2=0x7f914006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.742+0000 7f9168600640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f914006b520 0x7f914006d9e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9154009ad0 tx=0x7f9154008040 comp rx=0 tx=0).stop 2024-03-08T06:06:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.742+0000 7f9168600640 1 -- 172.21.15.40:0/2937892808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9160071ab0 msgr2=0x7f9160143d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.742+0000 7f9168600640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9160071ab0 0x7f9160143d10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c3a0 tx=0x7f914c00c860 comp rx=0 tx=0).stop 2024-03-08T06:06:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.743+0000 7f9168600640 1 -- 172.21.15.40:0/2937892808 shutdown_connections 2024-03-08T06:06:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.743+0000 7f9168600640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f914006b520 0x7f914006d9e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.743+0000 7f9168600640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9160071e90 0x7f91600b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.743+0000 7f9168600640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9160071ab0 0x7f9160143d10 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.743+0000 7f9168600640 1 --2- 172.21.15.40:0/2937892808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916006d600 0x7f916015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.743+0000 7f9168600640 1 -- 172.21.15.40:0/2937892808 >> 172.21.15.40:0/2937892808 conn(0x7f916006ae30 msgr2=0x7f916015f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.743+0000 7f9168600640 1 -- 172.21.15.40:0/2937892808 shutdown_connections 2024-03-08T06:06:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.743+0000 7f9168600640 1 -- 172.21.15.40:0/2937892808 wait complete. 2024-03-08T06:06:20.763 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:06:20.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.985+0000 7f5425800640 1 Processor -- start 2024-03-08T06:06:20.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.985+0000 7f5425800640 1 -- start start 2024-03-08T06:06:20.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.986+0000 7f5425800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f54200cf040 0x7f542006e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.986+0000 7f5425800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f54200b46c0 0x7f542015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.986+0000 7f5425800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5420071840 0x7f5420071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.986+0000 7f5425800640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542006f400 con 0x7f5420071840 2024-03-08T06:06:20.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.986+0000 7f5425800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54200b4bf0 con 0x7f54200cf040 2024-03-08T06:06:20.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.986+0000 7f5425800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542006d730 con 0x7f54200b46c0 2024-03-08T06:06:20.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.987+0000 7f541f400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5420071840 0x7f5420071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.987+0000 7f541f400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5420071840 0x7f5420071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39794/0 (socket says 172.21.15.40:39794) 2024-03-08T06:06:20.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.987+0000 7f541f400640 1 -- 172.21.15.40:0/1854084722 learned_addr learned my addr 172.21.15.40:0/1854084722 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:20.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.987+0000 7f541e000640 1 --2- 172.21.15.40:0/1854084722 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f54200b46c0 0x7f542015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.987+0000 7f541ea00640 1 --2- 172.21.15.40:0/1854084722 >> v2:172.21.15.53:3300/0 conn(0x7f54200cf040 0x7f542006e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.987+0000 7f541f400640 1 -- 172.21.15.40:0/1854084722 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f54200b46c0 msgr2=0x7f542015ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:20.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.987+0000 7f541f400640 1 --2- 172.21.15.40:0/1854084722 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f54200b46c0 0x7f542015ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.987+0000 7f541f400640 1 -- 172.21.15.40:0/1854084722 >> v2:172.21.15.53:3300/0 conn(0x7f54200cf040 msgr2=0x7f542006e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:20.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.987+0000 7f541f400640 1 --2- 172.21.15.40:0/1854084722 >> v2:172.21.15.53:3300/0 conn(0x7f54200cf040 0x7f542006e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.987+0000 7f541f400640 1 -- 172.21.15.40:0/1854084722 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542006da10 con 0x7f5420071840 2024-03-08T06:06:20.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.987+0000 7f541ea00640 1 --2- 172.21.15.40:0/1854084722 >> v2:172.21.15.53:3300/0 conn(0x7f54200cf040 0x7f542006e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:20.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.988+0000 7f541f400640 1 --2- 172.21.15.40:0/1854084722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5420071840 0x7f5420071cd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f54200b4690 tx=0x7f541802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc49c1d8abb4c92 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:20.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.989+0000 7f541d600640 1 -- 172.21.15.40:0/1854084722 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541802e070 con 0x7f5420071840 2024-03-08T06:06:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.989+0000 7f541d600640 1 -- 172.21.15.40:0/1854084722 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5418037070 con 0x7f5420071840 2024-03-08T06:06:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.989+0000 7f541d600640 1 -- 172.21.15.40:0/1854084722 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418031a00 con 0x7f5420071840 2024-03-08T06:06:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.989+0000 7f5425800640 1 -- 172.21.15.40:0/1854084722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5420071840 msgr2=0x7f5420071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.989+0000 7f5425800640 1 --2- 172.21.15.40:0/1854084722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5420071840 0x7f5420071cd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f54200b4690 tx=0x7f541802b880 comp rx=0 tx=0).stop 2024-03-08T06:06:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.989+0000 7f5425800640 1 -- 172.21.15.40:0/1854084722 shutdown_connections 2024-03-08T06:06:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.990+0000 7f5425800640 1 --2- 172.21.15.40:0/1854084722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5420071840 0x7f5420071cd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.990+0000 7f5425800640 1 --2- 172.21.15.40:0/1854084722 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f54200b46c0 0x7f542015ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.990+0000 7f5425800640 1 --2- 172.21.15.40:0/1854084722 >> v2:172.21.15.53:3300/0 conn(0x7f54200cf040 0x7f542006e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.990+0000 7f5425800640 1 -- 172.21.15.40:0/1854084722 >> 172.21.15.40:0/1854084722 conn(0x7f542006ae30 msgr2=0x7f542006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.990+0000 7f5425800640 1 -- 172.21.15.40:0/1854084722 shutdown_connections 2024-03-08T06:06:20.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.990+0000 7f5425800640 1 -- 172.21.15.40:0/1854084722 wait complete. 2024-03-08T06:06:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.991+0000 7f5425800640 1 Processor -- start 2024-03-08T06:06:20.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.991+0000 7f5425800640 1 -- start start 2024-03-08T06:06:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.992+0000 7f5425800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5420071840 0x7f5420133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.992+0000 7f5425800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54200b46c0 0x7f54200af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.992+0000 7f5425800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f54200cf040 0x7f54200b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.992+0000 7f5425800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5420146ab0 con 0x7f5420071840 2024-03-08T06:06:20.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.992+0000 7f5425800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5420146930 con 0x7f54200b46c0 2024-03-08T06:06:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.992+0000 7f5425800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5420146c30 con 0x7f54200cf040 2024-03-08T06:06:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5420071840 0x7f5420133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5420071840 0x7f5420133ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39802/0 (socket says 172.21.15.40:39802) 2024-03-08T06:06:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541ea00640 1 -- 172.21.15.40:0/1817458133 learned_addr learned my addr 172.21.15.40:0/1817458133 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541e000640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54200b46c0 0x7f54200af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541f400640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f54200cf040 0x7f54200b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541ea00640 1 -- 172.21.15.40:0/1817458133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f54200cf040 msgr2=0x7f54200b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541ea00640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f54200cf040 0x7f54200b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541ea00640 1 -- 172.21.15.40:0/1817458133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54200b46c0 msgr2=0x7f54200af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541ea00640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54200b46c0 0x7f54200af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541ea00640 1 -- 172.21.15.40:0/1817458133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200b00a0 con 0x7f5420071840 2024-03-08T06:06:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541f400640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f54200cf040 0x7f54200b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.993+0000 7f541e000640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54200b46c0 0x7f54200af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:06:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.994+0000 7f541ea00640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5420071840 0x7f5420133ec0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f541000c820 tx=0x7f541000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.994+0000 7f5416a00640 1 -- 172.21.15.40:0/1817458133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410015070 con 0x7f5420071840 2024-03-08T06:06:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.994+0000 7f5416a00640 1 -- 172.21.15.40:0/1817458133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5410007a30 con 0x7f5420071840 2024-03-08T06:06:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.994+0000 7f5416a00640 1 -- 172.21.15.40:0/1817458133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410007de0 con 0x7f5420071840 2024-03-08T06:06:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.994+0000 7f5425800640 1 -- 172.21.15.40:0/1817458133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54200b0380 con 0x7f5420071840 2024-03-08T06:06:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.994+0000 7f5425800640 1 -- 172.21.15.40:0/1817458133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54200b0940 con 0x7f5420071840 2024-03-08T06:06:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.995+0000 7f5425800640 1 -- 172.21.15.40:0/1817458133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5420067cb0 con 0x7f5420071840 2024-03-08T06:06:20.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.996+0000 7f5416a00640 1 -- 172.21.15.40:0/1817458133 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5410018070 con 0x7f5420071840 2024-03-08T06:06:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.997+0000 7f5416a00640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f540006b4d0 0x7f540006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.997+0000 7f5416a00640 1 -- 172.21.15.40:0/1817458133 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f541008a0a0 con 0x7f5420071840 2024-03-08T06:06:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.997+0000 7f541e000640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f540006b4d0 0x7f540006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:20.998+0000 7f541e000640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f540006b4d0 0x7f540006d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f54200acef0 tx=0x7f540c0039f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:21.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.002+0000 7f5416a00640 1 -- 172.21.15.40:0/1817458133 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5410058fb0 con 0x7f5420071840 2024-03-08T06:06:21.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.199+0000 7f5425800640 1 -- 172.21.15.40:0/1817458133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f542006ebd0 con 0x7f5420071840 2024-03-08T06:06:21.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.202+0000 7f5416a00640 1 -- 172.21.15.40:0/1817458133 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16585 (secure 0 0 0) 0x7f54100588b0 con 0x7f5420071840 2024-03-08T06:06:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:21.203 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:05:12.978712+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:05:09.544904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T06:05:10.468274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:06:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 -- 172.21.15.40:0/1817458133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f540006b4d0 msgr2=0x7f540006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f540006b4d0 0x7f540006d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f54200acef0 tx=0x7f540c0039f0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 -- 172.21.15.40:0/1817458133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5420071840 msgr2=0x7f5420133ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5420071840 0x7f5420133ec0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f541000c820 tx=0x7f541000cce0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 -- 172.21.15.40:0/1817458133 shutdown_connections 2024-03-08T06:06:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f540006b4d0 0x7f540006d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f54200cf040 0x7f54200b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54200b46c0 0x7f54200af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 --2- 172.21.15.40:0/1817458133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5420071840 0x7f5420133ec0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 -- 172.21.15.40:0/1817458133 >> 172.21.15.40:0/1817458133 conn(0x7f542006ae30 msgr2=0x7f542015fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 -- 172.21.15.40:0/1817458133 shutdown_connections 2024-03-08T06:06:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.205+0000 7f5425800640 1 -- 172.21.15.40:0/1817458133 wait complete. 2024-03-08T06:06:21.223 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-08T06:06:21.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:06:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.435+0000 7f17b1200640 1 Processor -- start 2024-03-08T06:06:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.435+0000 7f17b1200640 1 -- start start 2024-03-08T06:06:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.436+0000 7f17b1200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f17ac145520 0x7f17ac15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.436+0000 7f17b1200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ac15ceb0 con 0x7f17ac071df0 2024-03-08T06:06:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.436+0000 7f17b1200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ac15c330 con 0x7f17ac15bea0 2024-03-08T06:06:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.436+0000 7f17b1200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17ac15c610 con 0x7f17ac145520 2024-03-08T06:06:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.437+0000 7f17abe00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f17ac071df0 0x7f17ac06d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42014/0 (socket says 172.21.15.40:42014) 2024-03-08T06:06:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.437+0000 7f17abe00640 1 -- 172.21.15.40:0/512600792 learned_addr learned my addr 172.21.15.40:0/512600792 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.437+0000 7f17ab400640 1 --2- 172.21.15.40:0/512600792 >> v2:172.21.15.53:3300/0 conn(0x7f17ac145520 0x7f17ac15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.437+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889701577 0 0) 0x7f17ac15ceb0 con 0x7f17ac071df0 2024-03-08T06:06:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.437+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1790003520 con 0x7f17ac071df0 2024-03-08T06:06:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.437+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2461986811 0 0) 0x7f1790003520 con 0x7f17ac071df0 2024-03-08T06:06:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 >> v2:172.21.15.53:3300/0 conn(0x7f17ac145520 msgr2=0x7f17ac15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 --2- 172.21.15.40:0/512600792 >> v2:172.21.15.53:3300/0 conn(0x7f17ac145520 0x7f17ac15d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 >> v1:172.21.15.53:6789/0 conn(0x7f17ac15bea0 legacy=0x7f17ac0716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac15c8f0 con 0x7f17ac071df0 2024-03-08T06:06:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f17a40019f0 con 0x7f17ac071df0 2024-03-08T06:06:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 >> v1:172.21.15.40:6789/0 conn(0x7f17ac071df0 legacy=0x7f17ac06d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 --2- 172.21.15.40:0/512600792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17900039b0 0x7f1790003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 --2- 172.21.15.40:0/512600792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 0x7f1790004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 --2- 172.21.15.40:0/512600792 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f17900051c0 0x7f179000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac15c610 con 0x7f1790004800 2024-03-08T06:06:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179000c060 con 0x7f17900039b0 2024-03-08T06:06:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179000c260 con 0x7f17900051c0 2024-03-08T06:06:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aaa00640 1 --2- 172.21.15.40:0/512600792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 0x7f1790004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17ab400640 1 --2- 172.21.15.40:0/512600792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17900039b0 0x7f1790003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aaa00640 1 -- 172.21.15.40:0/512600792 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f17900051c0 msgr2=0x7f179000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aaa00640 1 --2- 172.21.15.40:0/512600792 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f17900051c0 0x7f179000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aaa00640 1 -- 172.21.15.40:0/512600792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17900039b0 msgr2=0x7f1790003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aaa00640 1 --2- 172.21.15.40:0/512600792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17900039b0 0x7f1790003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17aaa00640 1 -- 172.21.15.40:0/512600792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1790003520 con 0x7f1790004800 2024-03-08T06:06:21.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.438+0000 7f17ab400640 1 --2- 172.21.15.40:0/512600792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17900039b0 0x7f1790003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.439+0000 7f17abe00640 1 -- 172.21.15.40:0/512600792 reap_dead start 2024-03-08T06:06:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.439+0000 7f17aaa00640 1 --2- 172.21.15.40:0/512600792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 0x7f1790004c80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1798001750 tx=0x7f1790005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c26800756a53732 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.440+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798001370 con 0x7f1790004800 2024-03-08T06:06:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.440+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17980052d0 con 0x7f1790004800 2024-03-08T06:06:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.440+0000 7f17aa000640 1 -- 172.21.15.40:0/512600792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798003830 con 0x7f1790004800 2024-03-08T06:06:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.440+0000 7f17b1200640 1 -- 172.21.15.40:0/512600792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 msgr2=0x7f1790004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.440+0000 7f17b1200640 1 --2- 172.21.15.40:0/512600792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 0x7f1790004c80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1798001750 tx=0x7f1790005830 comp rx=0 tx=0).stop 2024-03-08T06:06:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.440+0000 7f17b1200640 1 -- 172.21.15.40:0/512600792 shutdown_connections 2024-03-08T06:06:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.440+0000 7f17b1200640 1 --2- 172.21.15.40:0/512600792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 0x7f1790004c80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.440+0000 7f17b1200640 1 -- 172.21.15.40:0/512600792 >> 172.21.15.40:0/512600792 conn(0x7f17ac0b1850 msgr2=0x7f17ac0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.440+0000 7f17b1200640 1 -- 172.21.15.40:0/512600792 shutdown_connections 2024-03-08T06:06:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.441+0000 7f17b1200640 1 -- 172.21.15.40:0/512600792 wait complete. 2024-03-08T06:06:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.442+0000 7f17b1200640 1 Processor -- start 2024-03-08T06:06:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.442+0000 7f17b1200640 1 -- start start 2024-03-08T06:06:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.443+0000 7f17b1200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 0x7f17ac15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.443+0000 7f17b1200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f17ac143fa0 0x7f17ac0b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.443+0000 7f17b1200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac0b47e0 0x7f17ac141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.443+0000 7f17b1200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac0af1c0 con 0x7f1790004800 2024-03-08T06:06:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.443+0000 7f17b1200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac0af040 con 0x7f17ac0b47e0 2024-03-08T06:06:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.443+0000 7f17b1200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac06e410 con 0x7f17ac143fa0 2024-03-08T06:06:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.444+0000 7f17ab400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 0x7f17ac15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.444+0000 7f17ab400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 0x7f17ac15b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39824/0 (socket says 172.21.15.40:39824) 2024-03-08T06:06:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.444+0000 7f17ab400640 1 -- 172.21.15.40:0/3905333451 learned_addr learned my addr 172.21.15.40:0/3905333451 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.444+0000 7f17aaa00640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f17ac143fa0 0x7f17ac0b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.444+0000 7f17abe00640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac0b47e0 0x7f17ac141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.444+0000 7f17ab400640 1 -- 172.21.15.40:0/3905333451 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f17ac143fa0 msgr2=0x7f17ac0b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.444+0000 7f17ab400640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f17ac143fa0 0x7f17ac0b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.444+0000 7f17ab400640 1 -- 172.21.15.40:0/3905333451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac0b47e0 msgr2=0x7f17ac141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.444+0000 7f17ab400640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac0b47e0 0x7f17ac141350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.444+0000 7f17ab400640 1 -- 172.21.15.40:0/3905333451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac0b3200 con 0x7f1790004800 2024-03-08T06:06:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.444+0000 7f17abe00640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac0b47e0 0x7f17ac141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.445+0000 7f17ab400640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 0x7f17ac15b210 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f179c0109d0 tx=0x7f179c010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.445+0000 7f17a3400640 1 -- 172.21.15.40:0/3905333451 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c00f040 con 0x7f1790004800 2024-03-08T06:06:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.445+0000 7f17a3400640 1 -- 172.21.15.40:0/3905333451 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c00ec20 con 0x7f1790004800 2024-03-08T06:06:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.445+0000 7f17b1200640 1 -- 172.21.15.40:0/3905333451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ac0b34e0 con 0x7f1790004800 2024-03-08T06:06:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.445+0000 7f17a3400640 1 -- 172.21.15.40:0/3905333451 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c01c400 con 0x7f1790004800 2024-03-08T06:06:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.445+0000 7f17b1200640 1 -- 172.21.15.40:0/3905333451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ac0b3aa0 con 0x7f1790004800 2024-03-08T06:06:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.446+0000 7f17b1200640 1 -- 172.21.15.40:0/3905333451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ac075100 con 0x7f1790004800 2024-03-08T06:06:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.447+0000 7f17a3400640 1 -- 172.21.15.40:0/3905333451 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f179c01c5a0 con 0x7f1790004800 2024-03-08T06:06:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.448+0000 7f17a3400640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f178c06b500 0x7f178c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.448+0000 7f17a3400640 1 -- 172.21.15.40:0/3905333451 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f179c08aa00 con 0x7f1790004800 2024-03-08T06:06:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.448+0000 7f17aaa00640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f178c06b500 0x7f178c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.449+0000 7f17aaa00640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f178c06b500 0x7f178c06d9c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1798033000 tx=0x7f1798009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.452+0000 7f17a3400640 1 -- 172.21.15.40:0/3905333451 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f179c018030 con 0x7f1790004800 2024-03-08T06:06:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.688+0000 7f17b1200640 1 -- 172.21.15.40:0/3905333451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17ac067ec0 con 0x7f1790004800 2024-03-08T06:06:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.690+0000 7f17a3400640 1 -- 172.21.15.40:0/3905333451 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f179c0597e0 con 0x7f1790004800 2024-03-08T06:06:21.690 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:21.690 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/2194146146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6835","nonce":2194146146},{"type":"v1","addr":"172.21.15.40:6837","nonce":2194146146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/831964898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":831964898},{"type":"v1","addr":"172.21.15.53:6835","nonce":831964898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4723,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2951334634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2951334634},{"type":"v1","addr":"172.21.15.53:6833","nonce":2951334634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:05:11.446139+0000","modified":"2024-03-08T06:05:14.485839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6836/1742457959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1742457959},{"type":"v1","addr":"172.21.15.40:6836","nonce":1742457959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:06:21.691 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2024-03-08T06:06:21.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 -- 172.21.15.40:0/3905333451 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f178c06b500 msgr2=0x7f178c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f178c06b500 0x7f178c06d9c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1798033000 tx=0x7f1798009030 comp rx=0 tx=0).stop 2024-03-08T06:06:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 -- 172.21.15.40:0/3905333451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 msgr2=0x7f17ac15b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 0x7f17ac15b210 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f179c0109d0 tx=0x7f179c010e90 comp rx=0 tx=0).stop 2024-03-08T06:06:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 -- 172.21.15.40:0/3905333451 shutdown_connections 2024-03-08T06:06:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f178c06b500 0x7f178c06d9c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac0b47e0 0x7f17ac141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f17ac143fa0 0x7f17ac0b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 --2- 172.21.15.40:0/3905333451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1790004800 0x7f17ac15b210 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 -- 172.21.15.40:0/3905333451 >> 172.21.15.40:0/3905333451 conn(0x7f17ac0b1850 msgr2=0x7f17ac0ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 -- 172.21.15.40:0/3905333451 shutdown_connections 2024-03-08T06:06:21.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.692+0000 7f17b1200640 1 -- 172.21.15.40:0/3905333451 wait complete. 2024-03-08T06:06:21.710 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:06:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.918+0000 7f548b000640 1 Processor -- start 2024-03-08T06:06:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.918+0000 7f548b000640 1 -- start start 2024-03-08T06:06:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.920+0000 7f548b000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484145520 0x7f548415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.920+0000 7f548b000640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5484071df0 0x7f548406d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.920+0000 7f548b000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548415ceb0 con 0x7f548415bea0 2024-03-08T06:06:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.920+0000 7f548b000640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548415c330 con 0x7f5484071df0 2024-03-08T06:06:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.920+0000 7f548b000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548415c610 con 0x7f5484145520 2024-03-08T06:06:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.920+0000 7f548a600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5484071df0 0x7f548406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.920+0000 7f5489c00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484145520 0x7f548415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.920+0000 7f548a600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5484071df0 0x7f548406d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:40538/0 (socket says 172.21.15.40:40538) 2024-03-08T06:06:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.920+0000 7f548a600640 1 -- 172.21.15.40:0/3807571808 learned_addr learned my addr 172.21.15.40:0/3807571808 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.921+0000 7f5489c00640 1 -- 172.21.15.40:0/3807571808 >> v2:172.21.15.53:3300/0 conn(0x7f5484071df0 msgr2=0x7f548406d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.921+0000 7f5489c00640 1 --2- 172.21.15.40:0/3807571808 >> v2:172.21.15.53:3300/0 conn(0x7f5484071df0 0x7f548406d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.921+0000 7f5489c00640 1 -- 172.21.15.40:0/3807571808 >> v1:172.21.15.53:6789/0 conn(0x7f548415bea0 legacy=0x7f54840716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.921+0000 7f5489c00640 1 -- 172.21.15.40:0/3807571808 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548415c8f0 con 0x7f5484145520 2024-03-08T06:06:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.921+0000 7f548a600640 1 --2- 172.21.15.40:0/3807571808 >> v2:172.21.15.53:3300/0 conn(0x7f5484071df0 0x7f548406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:06:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.923+0000 7f5489c00640 1 --2- 172.21.15.40:0/3807571808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484145520 0x7f548415d0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5478009d90 tx=0x7f547802beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7748f3774fae49a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.923+0000 7f5483e00640 1 -- 172.21.15.40:0/3807571808 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5478039070 con 0x7f5484145520 2024-03-08T06:06:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.923+0000 7f5483e00640 1 -- 172.21.15.40:0/3807571808 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547802cc60 con 0x7f5484145520 2024-03-08T06:06:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.923+0000 7f5483e00640 1 -- 172.21.15.40:0/3807571808 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5478034660 con 0x7f5484145520 2024-03-08T06:06:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.924+0000 7f548b000640 1 -- 172.21.15.40:0/3807571808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484145520 msgr2=0x7f548415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.924+0000 7f548b000640 1 --2- 172.21.15.40:0/3807571808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484145520 0x7f548415d0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5478009d90 tx=0x7f547802beb0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.924+0000 7f548b000640 1 -- 172.21.15.40:0/3807571808 shutdown_connections 2024-03-08T06:06:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.924+0000 7f548b000640 1 --2- 172.21.15.40:0/3807571808 >> v2:172.21.15.53:3300/0 conn(0x7f5484071df0 0x7f548406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.924+0000 7f548b000640 1 --2- 172.21.15.40:0/3807571808 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484145520 0x7f548415d0a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.924+0000 7f548b000640 1 -- 172.21.15.40:0/3807571808 >> 172.21.15.40:0/3807571808 conn(0x7f54840b1850 msgr2=0x7f54840b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:21.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.924+0000 7f548b000640 1 -- 172.21.15.40:0/3807571808 shutdown_connections 2024-03-08T06:06:21.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.924+0000 7f548b000640 1 -- 172.21.15.40:0/3807571808 wait complete. 2024-03-08T06:06:21.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.926+0000 7f548b000640 1 Processor -- start 2024-03-08T06:06:21.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.926+0000 7f548b000640 1 -- start start 2024-03-08T06:06:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.927+0000 7f548b000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484071df0 0x7f5484138850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.927+0000 7f548b000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5484145520 0x7f548413f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.927+0000 7f548b000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54840ad800 0x7f54840adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.927+0000 7f548b000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840af1c0 con 0x7f54840ad800 2024-03-08T06:06:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.927+0000 7f548b000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840af040 con 0x7f5484145520 2024-03-08T06:06:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.927+0000 7f548b000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548406e410 con 0x7f5484071df0 2024-03-08T06:06:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.927+0000 7f548a600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54840ad800 0x7f54840adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.927+0000 7f548a600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54840ad800 0x7f54840adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39832/0 (socket says 172.21.15.40:39832) 2024-03-08T06:06:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.927+0000 7f548a600640 1 -- 172.21.15.40:0/3580387261 learned_addr learned my addr 172.21.15.40:0/3580387261 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.927+0000 7f5489200640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5484145520 0x7f548413f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.928+0000 7f5489c00640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484071df0 0x7f5484138850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.928+0000 7f548a600640 1 -- 172.21.15.40:0/3580387261 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484071df0 msgr2=0x7f5484138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.928+0000 7f548a600640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484071df0 0x7f5484138850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.928+0000 7f548a600640 1 -- 172.21.15.40:0/3580387261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5484145520 msgr2=0x7f548413f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.928+0000 7f548a600640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5484145520 0x7f548413f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.928+0000 7f548a600640 1 -- 172.21.15.40:0/3580387261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548414ff50 con 0x7f54840ad800 2024-03-08T06:06:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.928+0000 7f5489c00640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484071df0 0x7f5484138850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:06:21.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.928+0000 7f548a600640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54840ad800 0x7f54840adcb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f547c00e930 tx=0x7f547c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:21.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.928+0000 7f5481600640 1 -- 172.21.15.40:0/3580387261 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c017070 con 0x7f54840ad800 2024-03-08T06:06:21.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.928+0000 7f5481600640 1 -- 172.21.15.40:0/3580387261 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c00ca00 con 0x7f54840ad800 2024-03-08T06:06:21.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.928+0000 7f5481600640 1 -- 172.21.15.40:0/3580387261 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c00cdd0 con 0x7f54840ad800 2024-03-08T06:06:21.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.929+0000 7f548b000640 1 -- 172.21.15.40:0/3580387261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548414fc70 con 0x7f54840ad800 2024-03-08T06:06:21.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.929+0000 7f548b000640 1 -- 172.21.15.40:0/3580387261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548413e980 con 0x7f54840ad800 2024-03-08T06:06:21.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.930+0000 7f548b000640 1 -- 172.21.15.40:0/3580387261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54840cdcf0 con 0x7f54840ad800 2024-03-08T06:06:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.931+0000 7f5481600640 1 -- 172.21.15.40:0/3580387261 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f547c013470 con 0x7f54840ad800 2024-03-08T06:06:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.931+0000 7f5481600640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f546406b500 0x7f546406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:21.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.932+0000 7f5481600640 1 -- 172.21.15.40:0/3580387261 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f547c089cf0 con 0x7f54840ad800 2024-03-08T06:06:21.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.932+0000 7f5489c00640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f546406b500 0x7f546406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:21.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.932+0000 7f5489c00640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f546406b500 0x7f546406d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5478036000 tx=0x7f5478030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:21.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:21.936+0000 7f5481600640 1 -- 172.21.15.40:0/3580387261 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54840cdcf0 con 0x7f54840ad800 2024-03-08T06:06:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.171+0000 7f548b000640 1 -- 172.21.15.40:0/3580387261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5484075100 con 0x7f54840ad800 2024-03-08T06:06:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.173+0000 7f5481600640 1 -- 172.21.15.40:0/3580387261 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f547c058480 con 0x7f54840ad800 2024-03-08T06:06:22.173 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:22.173 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/2194146146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6835","nonce":2194146146},{"type":"v1","addr":"172.21.15.40:6837","nonce":2194146146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/831964898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":831964898},{"type":"v1","addr":"172.21.15.53:6835","nonce":831964898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4723,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2951334634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2951334634},{"type":"v1","addr":"172.21.15.53:6833","nonce":2951334634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T06:05:11.446139+0000","modified":"2024-03-08T06:05:14.485839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6836/1742457959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1742457959},{"type":"v1","addr":"172.21.15.40:6836","nonce":1742457959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T06:06:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2024-03-08T06:06:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.175+0000 7f548b000640 1 -- 172.21.15.40:0/3580387261 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f546406b500 msgr2=0x7f546406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.175+0000 7f548b000640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f546406b500 0x7f546406d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5478036000 tx=0x7f5478030000 comp rx=0 tx=0).stop 2024-03-08T06:06:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.175+0000 7f548b000640 1 -- 172.21.15.40:0/3580387261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54840ad800 msgr2=0x7f54840adcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.175+0000 7f548b000640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54840ad800 0x7f54840adcb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f547c00e930 tx=0x7f547c00edf0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.176+0000 7f548b000640 1 -- 172.21.15.40:0/3580387261 shutdown_connections 2024-03-08T06:06:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.176+0000 7f548b000640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f546406b500 0x7f546406d9c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.176+0000 7f548b000640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54840ad800 0x7f54840adcb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.176+0000 7f548b000640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5484145520 0x7f548413f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.176+0000 7f548b000640 1 --2- 172.21.15.40:0/3580387261 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5484071df0 0x7f5484138850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.176+0000 7f548b000640 1 -- 172.21.15.40:0/3580387261 >> 172.21.15.40:0/3580387261 conn(0x7f54840b1850 msgr2=0x7f548415f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.176+0000 7f548b000640 1 -- 172.21.15.40:0/3580387261 shutdown_connections 2024-03-08T06:06:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.176+0000 7f548b000640 1 -- 172.21.15.40:0/3580387261 wait complete. 2024-03-08T06:06:22.195 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:06:22.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.422+0000 7fe14ac00640 1 Processor -- start 2024-03-08T06:06:22.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.422+0000 7fe14ac00640 1 -- start start 2024-03-08T06:06:22.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.424+0000 7fe14ac00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe14415bea0 0x7fe14415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.424+0000 7fe14ac00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14415ceb0 con 0x7fe144071df0 2024-03-08T06:06:22.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.424+0000 7fe14ac00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14415c330 con 0x7fe144145520 2024-03-08T06:06:22.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.424+0000 7fe14ac00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14415c610 con 0x7fe14415bea0 2024-03-08T06:06:22.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.424+0000 7fe14a200640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe144071df0 0x7fe14406d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42028/0 (socket says 172.21.15.40:42028) 2024-03-08T06:06:22.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.424+0000 7fe14a200640 1 -- 172.21.15.40:0/1500735373 learned_addr learned my addr 172.21.15.40:0/1500735373 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:22.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525810043 0 0) 0x7fe14415ceb0 con 0x7fe144071df0 2024-03-08T06:06:22.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe12c003520 con 0x7fe144071df0 2024-03-08T06:06:22.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3065408240 0 0) 0x7fe12c003520 con 0x7fe144071df0 2024-03-08T06:06:22.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 >> v2:172.21.15.53:3300/0 conn(0x7fe14415bea0 msgr2=0x7fe14415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 --2- 172.21.15.40:0/1500735373 >> v2:172.21.15.53:3300/0 conn(0x7fe14415bea0 0x7fe14415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 >> v1:172.21.15.53:6789/0 conn(0x7fe144145520 legacy=0x7fe1440716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14415c8f0 con 0x7fe144071df0 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fe13c0019f0 con 0x7fe144071df0 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 >> v1:172.21.15.40:6789/0 conn(0x7fe144071df0 legacy=0x7fe14406d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe12c0039b0 0x7fe12c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe12c004800 0x7fe12c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe12c0052e0 0x7fe12c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14415c610 con 0x7fe12c0052e0 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c00c1e0 con 0x7fe12c0039b0 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.425+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c00c3e0 con 0x7fe12c004800 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.426+0000 7fe14a200640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe12c0052e0 0x7fe12c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.426+0000 7fe149800640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe12c0039b0 0x7fe12c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.426+0000 7fe148e00640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe12c004800 0x7fe12c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.426+0000 7fe14a200640 1 -- 172.21.15.40:0/1500735373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe12c004800 msgr2=0x7fe12c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:22.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.426+0000 7fe14a200640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe12c004800 0x7fe12c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.426+0000 7fe14a200640 1 -- 172.21.15.40:0/1500735373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe12c0039b0 msgr2=0x7fe12c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:22.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.426+0000 7fe14a200640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe12c0039b0 0x7fe12c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.426+0000 7fe14a200640 1 -- 172.21.15.40:0/1500735373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c003520 con 0x7fe12c0052e0 2024-03-08T06:06:22.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.426+0000 7fe149800640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe12c0039b0 0x7fe12c003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:22.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.426+0000 7fe148e00640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe12c004800 0x7fe12c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:22.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.427+0000 7fe14a200640 1 -- 172.21.15.40:0/1500735373 reap_dead start 2024-03-08T06:06:22.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.427+0000 7fe14a200640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe12c0052e0 0x7fe12c00b9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe144072850 tx=0x7fe12c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a29c8acd1ed599 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:22.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.428+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00a480 con 0x7fe12c0052e0 2024-03-08T06:06:22.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.428+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c034630 con 0x7fe12c0052e0 2024-03-08T06:06:22.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.428+0000 7fe143e00640 1 -- 172.21.15.40:0/1500735373 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c034a20 con 0x7fe12c0052e0 2024-03-08T06:06:22.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.428+0000 7fe14ac00640 1 -- 172.21.15.40:0/1500735373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe12c0052e0 msgr2=0x7fe12c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.428+0000 7fe14ac00640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe12c0052e0 0x7fe12c00b9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe144072850 tx=0x7fe12c005830 comp rx=0 tx=0).stop 2024-03-08T06:06:22.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.428+0000 7fe14ac00640 1 -- 172.21.15.40:0/1500735373 shutdown_connections 2024-03-08T06:06:22.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.428+0000 7fe14ac00640 1 --2- 172.21.15.40:0/1500735373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe12c0052e0 0x7fe12c00b9c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.428+0000 7fe14ac00640 1 -- 172.21.15.40:0/1500735373 >> 172.21.15.40:0/1500735373 conn(0x7fe1440b1850 msgr2=0x7fe1440b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:22.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.428+0000 7fe14ac00640 1 -- 172.21.15.40:0/1500735373 shutdown_connections 2024-03-08T06:06:22.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.428+0000 7fe14ac00640 1 -- 172.21.15.40:0/1500735373 wait complete. 2024-03-08T06:06:22.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.430+0000 7fe14ac00640 1 Processor -- start 2024-03-08T06:06:22.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.430+0000 7fe14ac00640 1 -- start start 2024-03-08T06:06:22.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.431+0000 7fe14ac00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe12c0052e0 0x7fe144137120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.431+0000 7fe14ac00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1440ae340 0x7fe1441398f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.431+0000 7fe14ac00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe144143b60 0x7fe144143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.431+0000 7fe14ac00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440af1c0 con 0x7fe1440ae340 2024-03-08T06:06:22.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.431+0000 7fe14ac00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440af040 con 0x7fe144143b60 2024-03-08T06:06:22.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.431+0000 7fe14ac00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14406e410 con 0x7fe12c0052e0 2024-03-08T06:06:22.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.431+0000 7fe148e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1440ae340 0x7fe1441398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.432+0000 7fe148e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1440ae340 0x7fe1441398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39854/0 (socket says 172.21.15.40:39854) 2024-03-08T06:06:22.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.432+0000 7fe148e00640 1 -- 172.21.15.40:0/3245956834 learned_addr learned my addr 172.21.15.40:0/3245956834 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:22.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.432+0000 7fe149800640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe12c0052e0 0x7fe144137120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.432+0000 7fe14a200640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe144143b60 0x7fe144143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.432+0000 7fe148e00640 1 -- 172.21.15.40:0/3245956834 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe12c0052e0 msgr2=0x7fe144137120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.432+0000 7fe148e00640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe12c0052e0 0x7fe144137120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.432+0000 7fe148e00640 1 -- 172.21.15.40:0/3245956834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe144143b60 msgr2=0x7fe144143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.432+0000 7fe148e00640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe144143b60 0x7fe144143fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.432+0000 7fe148e00640 1 -- 172.21.15.40:0/3245956834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14413a240 con 0x7fe1440ae340 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.432+0000 7fe148e00640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1440ae340 0x7fe1441398f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe13000c7f0 tx=0x7fe13000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.432+0000 7fe141600640 1 -- 172.21.15.40:0/3245956834 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130004230 con 0x7fe1440ae340 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.433+0000 7fe141600640 1 -- 172.21.15.40:0/3245956834 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1300043d0 con 0x7fe1440ae340 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.433+0000 7fe141600640 1 -- 172.21.15.40:0/3245956834 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130007a10 con 0x7fe1440ae340 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.433+0000 7fe14ac00640 1 -- 172.21.15.40:0/3245956834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14413a520 con 0x7fe1440ae340 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.433+0000 7fe14ac00640 1 -- 172.21.15.40:0/3245956834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14413aae0 con 0x7fe1440ae340 2024-03-08T06:06:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.434+0000 7fe14ac00640 1 -- 172.21.15.40:0/3245956834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe144075100 con 0x7fe1440ae340 2024-03-08T06:06:22.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.435+0000 7fe141600640 1 -- 172.21.15.40:0/3245956834 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe130007bb0 con 0x7fe1440ae340 2024-03-08T06:06:22.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.436+0000 7fe141600640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12406b4d0 0x7fe12406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.436+0000 7fe141600640 1 -- 172.21.15.40:0/3245956834 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe13008a260 con 0x7fe1440ae340 2024-03-08T06:06:22.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.436+0000 7fe149800640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12406b4d0 0x7fe12406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.437+0000 7fe149800640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12406b4d0 0x7fe12406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe1380015b0 tx=0x7fe138009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:22.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.440+0000 7fe141600640 1 -- 172.21.15.40:0/3245956834 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe130015020 con 0x7fe1440ae340 2024-03-08T06:06:22.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.640+0000 7fe14ac00640 1 -- 172.21.15.40:0/3245956834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe14415d930 con 0x7fe1440ae340 2024-03-08T06:06:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.642+0000 7fe141600640 1 -- 172.21.15.40:0/3245956834 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16585 (secure 0 0 0) 0x7fe1300590e0 con 0x7fe1440ae340 2024-03-08T06:06:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:22.643 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:05:12.978712+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T06:05:09.544904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T06:05:10.468274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.645+0000 7fe14ac00640 1 -- 172.21.15.40:0/3245956834 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12406b4d0 msgr2=0x7fe12406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.645+0000 7fe14ac00640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12406b4d0 0x7fe12406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe1380015b0 tx=0x7fe138009040 comp rx=0 tx=0).stop 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.645+0000 7fe14ac00640 1 -- 172.21.15.40:0/3245956834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1440ae340 msgr2=0x7fe1441398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.645+0000 7fe14ac00640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1440ae340 0x7fe1441398f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe13000c7f0 tx=0x7fe13000ccb0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.645+0000 7fe14ac00640 1 -- 172.21.15.40:0/3245956834 shutdown_connections 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.645+0000 7fe14ac00640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fe12406b4d0 0x7fe12406d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.645+0000 7fe14ac00640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe144143b60 0x7fe144143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.645+0000 7fe14ac00640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1440ae340 0x7fe1441398f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.645+0000 7fe14ac00640 1 --2- 172.21.15.40:0/3245956834 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe12c0052e0 0x7fe144137120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.645+0000 7fe14ac00640 1 -- 172.21.15.40:0/3245956834 >> 172.21.15.40:0/3245956834 conn(0x7fe1440b1850 msgr2=0x7fe14415e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.646+0000 7fe14ac00640 1 -- 172.21.15.40:0/3245956834 shutdown_connections 2024-03-08T06:06:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.646+0000 7fe14ac00640 1 -- 172.21.15.40:0/3245956834 wait complete. 2024-03-08T06:06:22.665 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-08T06:06:22.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.884+0000 7f8121800640 1 Processor -- start 2024-03-08T06:06:22.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.885+0000 7f8121800640 1 -- start start 2024-03-08T06:06:22.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.886+0000 7f8121800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f811c0cd730 0x7f811c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.886+0000 7f8121800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c0cc690 con 0x7f811c145250 2024-03-08T06:06:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.886+0000 7f8121800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c0cc970 con 0x7f811c0717d0 2024-03-08T06:06:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.886+0000 7f8121800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811c0ccc50 con 0x7f811c0cd730 2024-03-08T06:06:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.886+0000 7f811aa00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f811c145250 0x7f811c06eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42040/0 (socket says 172.21.15.40:42040) 2024-03-08T06:06:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.886+0000 7f811aa00640 1 -- 172.21.15.40:0/3675949907 learned_addr learned my addr 172.21.15.40:0/3675949907 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f811a000640 1 --2- 172.21.15.40:0/3675949907 >> v2:172.21.15.53:3300/0 conn(0x7f811c0cd730 0x7f811c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293579346 0 0) 0x7f811c0cc690 con 0x7f811c145250 2024-03-08T06:06:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8100003520 con 0x7f811c145250 2024-03-08T06:06:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 991771069 0 0) 0x7f8100003520 con 0x7f811c145250 2024-03-08T06:06:22.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 >> v2:172.21.15.53:3300/0 conn(0x7f811c0cd730 msgr2=0x7f811c0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 --2- 172.21.15.40:0/3675949907 >> v2:172.21.15.53:3300/0 conn(0x7f811c0cd730 0x7f811c0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 >> v1:172.21.15.53:6789/0 conn(0x7f811c0717d0 legacy=0x7f811c15c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c0ccf30 con 0x7f811c145250 2024-03-08T06:06:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f810c0019f0 con 0x7f811c145250 2024-03-08T06:06:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 >> v1:172.21.15.40:6789/0 conn(0x7f811c145250 legacy=0x7f811c06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 --2- 172.21.15.40:0/3675949907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8100003a00 0x7f8100003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 --2- 172.21.15.40:0/3675949907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8100004800 0x7f8100004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 --2- 172.21.15.40:0/3675949907 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8100005210 0x7f810000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c0ccc50 con 0x7f8100004800 2024-03-08T06:06:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810000c140 con 0x7f8100003a00 2024-03-08T06:06:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810000c340 con 0x7f8100005210 2024-03-08T06:06:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.887+0000 7f811a000640 1 --2- 172.21.15.40:0/3675949907 >> v2:172.21.15.53:3300/0 conn(0x7f811c0cd730 0x7f811c0cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:06:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.888+0000 7f811a000640 1 --2- 172.21.15.40:0/3675949907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8100004800 0x7f8100004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.888+0000 7f811aa00640 1 --2- 172.21.15.40:0/3675949907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8100003a00 0x7f8100003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.888+0000 7f811a000640 1 -- 172.21.15.40:0/3675949907 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8100005210 msgr2=0x7f810000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.888+0000 7f811a000640 1 --2- 172.21.15.40:0/3675949907 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8100005210 0x7f810000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.888+0000 7f811a000640 1 -- 172.21.15.40:0/3675949907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8100003a00 msgr2=0x7f8100003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.888+0000 7f811a000640 1 --2- 172.21.15.40:0/3675949907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8100003a00 0x7f8100003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.888+0000 7f811a000640 1 -- 172.21.15.40:0/3675949907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8100003520 con 0x7f8100004800 2024-03-08T06:06:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.888+0000 7f811b400640 1 -- 172.21.15.40:0/3675949907 reap_dead start 2024-03-08T06:06:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.889+0000 7f811a000640 1 --2- 172.21.15.40:0/3675949907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8100004800 0x7f8100004c80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8104006fd0 tx=0x7f8100005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca76a202bb357c63 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.889+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810400f040 con 0x7f8100004800 2024-03-08T06:06:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.890+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8104004230 con 0x7f8100004800 2024-03-08T06:06:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.890+0000 7f8119600640 1 -- 172.21.15.40:0/3675949907 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104003610 con 0x7f8100004800 2024-03-08T06:06:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.890+0000 7f8121800640 1 -- 172.21.15.40:0/3675949907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8100004800 msgr2=0x7f8100004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.890+0000 7f8121800640 1 --2- 172.21.15.40:0/3675949907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8100004800 0x7f8100004c80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8104006fd0 tx=0x7f8100005830 comp rx=0 tx=0).stop 2024-03-08T06:06:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.890+0000 7f8121800640 1 -- 172.21.15.40:0/3675949907 shutdown_connections 2024-03-08T06:06:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.890+0000 7f8121800640 1 --2- 172.21.15.40:0/3675949907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8100004800 0x7f8100004c80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.890+0000 7f8121800640 1 -- 172.21.15.40:0/3675949907 >> 172.21.15.40:0/3675949907 conn(0x7f811c06ae30 msgr2=0x7f811c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.890+0000 7f8121800640 1 -- 172.21.15.40:0/3675949907 shutdown_connections 2024-03-08T06:06:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.890+0000 7f8121800640 1 -- 172.21.15.40:0/3675949907 wait complete. 2024-03-08T06:06:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.892+0000 7f8121800640 1 Processor -- start 2024-03-08T06:06:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.892+0000 7f8121800640 1 -- start start 2024-03-08T06:06:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.893+0000 7f8121800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8100004800 0x7f811c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.893+0000 7f8121800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f811c15dbb0 0x7f811c141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.893+0000 7f8121800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c1416a0 0x7f811c1f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.893+0000 7f8121800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c1457b0 con 0x7f811c1416a0 2024-03-08T06:06:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.893+0000 7f8121800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c145630 con 0x7f811c15dbb0 2024-03-08T06:06:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.893+0000 7f8121800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c0cc0b0 con 0x7f8100004800 2024-03-08T06:06:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.893+0000 7f811b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c1416a0 0x7f811c1f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.894+0000 7f811b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c1416a0 0x7f811c1f30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39876/0 (socket says 172.21.15.40:39876) 2024-03-08T06:06:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.894+0000 7f811b400640 1 -- 172.21.15.40:0/3355014074 learned_addr learned my addr 172.21.15.40:0/3355014074 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.894+0000 7f811a000640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f811c15dbb0 0x7f811c141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.894+0000 7f811aa00640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8100004800 0x7f811c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.894+0000 7f811b400640 1 -- 172.21.15.40:0/3355014074 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8100004800 msgr2=0x7f811c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.894+0000 7f811b400640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8100004800 0x7f811c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.894+0000 7f811b400640 1 -- 172.21.15.40:0/3355014074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f811c15dbb0 msgr2=0x7f811c141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.894+0000 7f811b400640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f811c15dbb0 0x7f811c141160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.894+0000 7f811b400640 1 -- 172.21.15.40:0/3355014074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c15e150 con 0x7f811c1416a0 2024-03-08T06:06:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.894+0000 7f811b400640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c1416a0 0x7f811c1f30a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f811c0a84d0 tx=0x7f811400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.894+0000 7f8112a00640 1 -- 172.21.15.40:0/3355014074 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114017070 con 0x7f811c1416a0 2024-03-08T06:06:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.895+0000 7f8112a00640 1 -- 172.21.15.40:0/3355014074 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81140072d0 con 0x7f811c1416a0 2024-03-08T06:06:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.895+0000 7f8112a00640 1 -- 172.21.15.40:0/3355014074 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81140057c0 con 0x7f811c1416a0 2024-03-08T06:06:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.895+0000 7f8121800640 1 -- 172.21.15.40:0/3355014074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c15e430 con 0x7f811c1416a0 2024-03-08T06:06:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.895+0000 7f8121800640 1 -- 172.21.15.40:0/3355014074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811c156c00 con 0x7f811c1416a0 2024-03-08T06:06:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.896+0000 7f8121800640 1 -- 172.21.15.40:0/3355014074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811c141b20 con 0x7f811c1416a0 2024-03-08T06:06:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.897+0000 7f8112a00640 1 -- 172.21.15.40:0/3355014074 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f811401a070 con 0x7f811c1416a0 2024-03-08T06:06:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.898+0000 7f8112a00640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f80f806b500 0x7f80f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.898+0000 7f8112a00640 1 -- 172.21.15.40:0/3355014074 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8114089230 con 0x7f811c1416a0 2024-03-08T06:06:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.898+0000 7f811aa00640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f80f806b500 0x7f80f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.899+0000 7f811aa00640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f80f806b500 0x7f80f806d9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f810c000f80 tx=0x7f810c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:22.903+0000 7f8112a00640 1 -- 172.21.15.40:0/3355014074 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81140580a0 con 0x7f811c1416a0 2024-03-08T06:06:23.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.145+0000 7f8121800640 1 -- 172.21.15.40:0/3355014074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f811c067cb0 con 0x7f811c1416a0 2024-03-08T06:06:23.146 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-03-08T06:06:23.147+0000 7fce8d000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-08T06:06:23.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.150+0000 7f8112a00640 1 -- 172.21.15.40:0/3355014074 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8114007d40 con 0x7f811c1416a0 2024-03-08T06:06:23.150 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-08T06:06:23.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.153+0000 7f8121800640 1 -- 172.21.15.40:0/3355014074 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f80f806b500 msgr2=0x7f80f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:23.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.153+0000 7f8121800640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f80f806b500 0x7f80f806d9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f810c000f80 tx=0x7f810c009040 comp rx=0 tx=0).stop 2024-03-08T06:06:23.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.153+0000 7f8121800640 1 -- 172.21.15.40:0/3355014074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c1416a0 msgr2=0x7f811c1f30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:23.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.153+0000 7f8121800640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c1416a0 0x7f811c1f30a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f811c0a84d0 tx=0x7f811400c7a0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.153+0000 7f8121800640 1 -- 172.21.15.40:0/3355014074 shutdown_connections 2024-03-08T06:06:23.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.153+0000 7f8121800640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f80f806b500 0x7f80f806d9c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.153+0000 7f8121800640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f811c1416a0 0x7f811c1f30a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.153+0000 7f8121800640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f811c15dbb0 0x7f811c141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.153+0000 7f8121800640 1 --2- 172.21.15.40:0/3355014074 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8100004800 0x7f811c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.153+0000 7f8121800640 1 -- 172.21.15.40:0/3355014074 >> 172.21.15.40:0/3355014074 conn(0x7f811c06ae30 msgr2=0x7f811c06e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:23.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.155+0000 7f8121800640 1 -- 172.21.15.40:0/3355014074 shutdown_connections 2024-03-08T06:06:23.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.155+0000 7f8121800640 1 -- 172.21.15.40:0/3355014074 wait complete. 2024-03-08T06:06:23.174 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-08T06:06:23.255 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-03-08T06:06:23.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.367+0000 7fbb6a400640 1 Processor -- start 2024-03-08T06:06:23.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.367+0000 7fbb6a400640 1 -- start start 2024-03-08T06:06:23.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.369+0000 7fbb6a400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb641455b0 0x7fbb6415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:23.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.369+0000 7fbb6a400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbb641604f0 0x7fbb640cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:23.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.369+0000 7fbb6a400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb64071950 0x7fbb64071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:23.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.369+0000 7fbb6a400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb640cdc20 con 0x7fbb64071950 2024-03-08T06:06:23.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.369+0000 7fbb6a400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb640722c0 con 0x7fbb641604f0 2024-03-08T06:06:23.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.369+0000 7fbb6a400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640cd5a0 con 0x7fbb641455b0 2024-03-08T06:06:23.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.369+0000 7fbb63e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb64071950 0x7fbb64071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:23.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.369+0000 7fbb63e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb64071950 0x7fbb64071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39910/0 (socket says 172.21.15.40:39910) 2024-03-08T06:06:23.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.369+0000 7fbb63e00640 1 -- 172.21.15.40:0/3524806444 learned_addr learned my addr 172.21.15.40:0/3524806444 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:23.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.369+0000 7fbb63400640 1 --2- 172.21.15.40:0/3524806444 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb641455b0 0x7fbb6415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:23.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.369+0000 7fbb62a00640 1 --2- 172.21.15.40:0/3524806444 >> v2:172.21.15.53:3300/0 conn(0x7fbb641604f0 0x7fbb640cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:23.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.370+0000 7fbb63e00640 1 -- 172.21.15.40:0/3524806444 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb641455b0 msgr2=0x7fbb6415c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:23.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.370+0000 7fbb63e00640 1 --2- 172.21.15.40:0/3524806444 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb641455b0 0x7fbb6415c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.370+0000 7fbb63e00640 1 -- 172.21.15.40:0/3524806444 >> v2:172.21.15.53:3300/0 conn(0x7fbb641604f0 msgr2=0x7fbb640cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:23.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.370+0000 7fbb63e00640 1 --2- 172.21.15.40:0/3524806444 >> v2:172.21.15.53:3300/0 conn(0x7fbb641604f0 0x7fbb640cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.370+0000 7fbb63e00640 1 -- 172.21.15.40:0/3524806444 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640ccfe0 con 0x7fbb64071950 2024-03-08T06:06:23.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.370+0000 7fbb63400640 1 --2- 172.21.15.40:0/3524806444 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb641455b0 0x7fbb6415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:06:23.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.371+0000 7fbb63e00640 1 --2- 172.21.15.40:0/3524806444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb64071950 0x7fbb64071d80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbb640b5080 tx=0x7fbb5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11849b5fec0230b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:23.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.371+0000 7fbb62000640 1 -- 172.21.15.40:0/3524806444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c035020 con 0x7fbb64071950 2024-03-08T06:06:23.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.371+0000 7fbb62000640 1 -- 172.21.15.40:0/3524806444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c031870 con 0x7fbb64071950 2024-03-08T06:06:23.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.371+0000 7fbb62000640 1 -- 172.21.15.40:0/3524806444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c031c40 con 0x7fbb64071950 2024-03-08T06:06:23.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.372+0000 7fbb6a400640 1 -- 172.21.15.40:0/3524806444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb64071950 msgr2=0x7fbb64071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:23.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.372+0000 7fbb6a400640 1 --2- 172.21.15.40:0/3524806444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb64071950 0x7fbb64071d80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbb640b5080 tx=0x7fbb5c02b910 comp rx=0 tx=0).stop 2024-03-08T06:06:23.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.372+0000 7fbb6a400640 1 -- 172.21.15.40:0/3524806444 shutdown_connections 2024-03-08T06:06:23.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.372+0000 7fbb6a400640 1 --2- 172.21.15.40:0/3524806444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb64071950 0x7fbb64071d80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.372+0000 7fbb6a400640 1 --2- 172.21.15.40:0/3524806444 >> v2:172.21.15.53:3300/0 conn(0x7fbb641604f0 0x7fbb640cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.372+0000 7fbb6a400640 1 --2- 172.21.15.40:0/3524806444 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb641455b0 0x7fbb6415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.372+0000 7fbb6a400640 1 -- 172.21.15.40:0/3524806444 >> 172.21.15.40:0/3524806444 conn(0x7fbb6406ae30 msgr2=0x7fbb6406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:23.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.372+0000 7fbb6a400640 1 -- 172.21.15.40:0/3524806444 shutdown_connections 2024-03-08T06:06:23.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.372+0000 7fbb6a400640 1 -- 172.21.15.40:0/3524806444 wait complete. 2024-03-08T06:06:23.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.374+0000 7fbb6a400640 1 Processor -- start 2024-03-08T06:06:23.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.374+0000 7fbb6a400640 1 -- start start 2024-03-08T06:06:23.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.375+0000 7fbb6a400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb64071950 0x7fbb6413d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.375+0000 7fbb6a400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb641455b0 0x7fbb6413fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.375+0000 7fbb6a400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb641604f0 0x7fbb6415b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.375+0000 7fbb6a400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6415c710 con 0x7fbb64071950 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.375+0000 7fbb6a400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640cd8b0 con 0x7fbb641604f0 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.375+0000 7fbb6a400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64145990 con 0x7fbb641455b0 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.375+0000 7fbb63400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb64071950 0x7fbb6413d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.376+0000 7fbb63400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb64071950 0x7fbb6413d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39926/0 (socket says 172.21.15.40:39926) 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.376+0000 7fbb63400640 1 -- 172.21.15.40:0/1511966339 learned_addr learned my addr 172.21.15.40:0/1511966339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.376+0000 7fbb63400640 1 -- 172.21.15.40:0/1511966339 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb641455b0 msgr2=0x7fbb6413fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.376+0000 7fbb63400640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb641455b0 0x7fbb6413fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.376+0000 7fbb63400640 1 -- 172.21.15.40:0/1511966339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb641604f0 msgr2=0x7fbb6415b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.376+0000 7fbb63400640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb641604f0 0x7fbb6415b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.376+0000 7fbb63400640 1 -- 172.21.15.40:0/1511966339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6413e920 con 0x7fbb64071950 2024-03-08T06:06:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.376+0000 7fbb63400640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb64071950 0x7fbb6413d670 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbb54007ef0 tx=0x7fbb5400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:23.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.376+0000 7fbb5b400640 1 -- 172.21.15.40:0/1511966339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5401a070 con 0x7fbb64071950 2024-03-08T06:06:23.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.377+0000 7fbb5b400640 1 -- 172.21.15.40:0/1511966339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb54003b90 con 0x7fbb64071950 2024-03-08T06:06:23.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.377+0000 7fbb5b400640 1 -- 172.21.15.40:0/1511966339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54004030 con 0x7fbb64071950 2024-03-08T06:06:23.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.377+0000 7fbb6a400640 1 -- 172.21.15.40:0/1511966339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6413dda0 con 0x7fbb64071950 2024-03-08T06:06:23.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.377+0000 7fbb6a400640 1 -- 172.21.15.40:0/1511966339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6413e360 con 0x7fbb64071950 2024-03-08T06:06:23.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.378+0000 7fbb6a400640 1 -- 172.21.15.40:0/1511966339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb64067cb0 con 0x7fbb64071950 2024-03-08T06:06:23.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.379+0000 7fbb5b400640 1 -- 172.21.15.40:0/1511966339 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb5401d070 con 0x7fbb64071950 2024-03-08T06:06:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.380+0000 7fbb5b400640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbb4006b4d0 0x7fbb4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.380+0000 7fbb5b400640 1 -- 172.21.15.40:0/1511966339 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbb5408a450 con 0x7fbb64071950 2024-03-08T06:06:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.380+0000 7fbb62a00640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbb4006b4d0 0x7fbb4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.381+0000 7fbb62a00640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbb4006b4d0 0x7fbb4006d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c006fd0 tx=0x7fbb4c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:23.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.384+0000 7fbb5b400640 1 -- 172.21.15.40:0/1511966339 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb54010070 con 0x7fbb64071950 2024-03-08T06:06:23.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:23.620+0000 7fbb6a400640 1 -- 172.21.15.40:0/1511966339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbb641f1c30 con 0x7fbb64071950 2024-03-08T06:06:24.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.154+0000 7fbb5b400640 1 -- 172.21.15.40:0/1511966339 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbb54058b80 con 0x7fbb64071950 2024-03-08T06:06:24.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.157+0000 7fbb6a400640 1 -- 172.21.15.40:0/1511966339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbb4006b4d0 msgr2=0x7fbb4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:24.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.157+0000 7fbb6a400640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbb4006b4d0 0x7fbb4006d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c006fd0 tx=0x7fbb4c008040 comp rx=0 tx=0).stop 2024-03-08T06:06:24.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.157+0000 7fbb6a400640 1 -- 172.21.15.40:0/1511966339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb64071950 msgr2=0x7fbb6413d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:24.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.157+0000 7fbb6a400640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb64071950 0x7fbb6413d670 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbb54007ef0 tx=0x7fbb5400d4a0 comp rx=0 tx=0).stop 2024-03-08T06:06:24.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.158+0000 7fbb6a400640 1 -- 172.21.15.40:0/1511966339 shutdown_connections 2024-03-08T06:06:24.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.158+0000 7fbb6a400640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbb4006b4d0 0x7fbb4006d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:24.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.158+0000 7fbb6a400640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb641604f0 0x7fbb6415b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:24.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.158+0000 7fbb6a400640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbb641455b0 0x7fbb6413fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:24.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.158+0000 7fbb6a400640 1 --2- 172.21.15.40:0/1511966339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb64071950 0x7fbb6413d670 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:24.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.158+0000 7fbb6a400640 1 -- 172.21.15.40:0/1511966339 >> 172.21.15.40:0/1511966339 conn(0x7fbb6406ae30 msgr2=0x7fbb6406b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:24.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.158+0000 7fbb6a400640 1 -- 172.21.15.40:0/1511966339 shutdown_connections 2024-03-08T06:06:24.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.158+0000 7fbb6a400640 1 -- 172.21.15.40:0/1511966339 wait complete. 2024-03-08T06:06:24.182 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-08T06:06:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.407+0000 7f303f000640 1 Processor -- start 2024-03-08T06:06:24.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.407+0000 7f303f000640 1 -- start start 2024-03-08T06:06:24.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.409+0000 7f303f000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f303815de30 0x7f30380b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:24.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.409+0000 7f303f000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f30380d3610 0x7f30380d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:24.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.409+0000 7f303f000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30380d2570 con 0x7f30380d0a00 2024-03-08T06:06:24.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.409+0000 7f303f000640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303815e210 con 0x7f30380d3610 2024-03-08T06:06:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.409+0000 7f303f000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30380d2850 con 0x7f303815de30 2024-03-08T06:06:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.409+0000 7f303d200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f30380d3610 0x7f30380d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.409+0000 7f303d200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f30380d3610 0x7f30380d1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39928/0 (socket says 172.21.15.40:39928) 2024-03-08T06:06:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.409+0000 7f303d200640 1 -- 172.21.15.40:0/1172444277 learned_addr learned my addr 172.21.15.40:0/1172444277 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.409+0000 7f303dc00640 1 --2- 172.21.15.40:0/1172444277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f303815de30 0x7f30380b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.410+0000 7f303d200640 1 -- 172.21.15.40:0/1172444277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f303815de30 msgr2=0x7f30380b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.410+0000 7f303d200640 1 --2- 172.21.15.40:0/1172444277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f303815de30 0x7f30380b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.410+0000 7f303d200640 1 -- 172.21.15.40:0/1172444277 >> v1:172.21.15.53:6789/0 conn(0x7f30380d0a00 legacy=0x7f30380b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.410+0000 7f303d200640 1 -- 172.21.15.40:0/1172444277 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30380d2b30 con 0x7f30380d3610 2024-03-08T06:06:24.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.411+0000 7f303d200640 1 --2- 172.21.15.40:0/1172444277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30380d3610 0x7f30380d1d20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3024009870 tx=0x7f302402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9697fd884eae88f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:24.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.411+0000 7f3037e00640 1 -- 172.21.15.40:0/1172444277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024035020 con 0x7f30380d3610 2024-03-08T06:06:24.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.411+0000 7f3037e00640 1 -- 172.21.15.40:0/1172444277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3024031640 con 0x7f30380d3610 2024-03-08T06:06:24.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.411+0000 7f3037e00640 1 -- 172.21.15.40:0/1172444277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302402d040 con 0x7f30380d3610 2024-03-08T06:06:24.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.412+0000 7f303f000640 1 -- 172.21.15.40:0/1172444277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30380d3610 msgr2=0x7f30380d1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:24.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.412+0000 7f303f000640 1 --2- 172.21.15.40:0/1172444277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30380d3610 0x7f30380d1d20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3024009870 tx=0x7f302402b8f0 comp rx=0 tx=0).stop 2024-03-08T06:06:24.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.412+0000 7f303f000640 1 -- 172.21.15.40:0/1172444277 shutdown_connections 2024-03-08T06:06:24.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.412+0000 7f303f000640 1 --2- 172.21.15.40:0/1172444277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30380d3610 0x7f30380d1d20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:24.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.412+0000 7f303f000640 1 --2- 172.21.15.40:0/1172444277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f303815de30 0x7f30380b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:24.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.412+0000 7f303f000640 1 -- 172.21.15.40:0/1172444277 >> 172.21.15.40:0/1172444277 conn(0x7f303806c740 msgr2=0x7f303806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:24.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.412+0000 7f303f000640 1 -- 172.21.15.40:0/1172444277 shutdown_connections 2024-03-08T06:06:24.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.412+0000 7f303f000640 1 -- 172.21.15.40:0/1172444277 wait complete. 2024-03-08T06:06:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.414+0000 7f303f000640 1 Processor -- start 2024-03-08T06:06:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.414+0000 7f303f000640 1 -- start start 2024-03-08T06:06:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.415+0000 7f303f000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380d0a00 0x7f30381f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.415+0000 7f303f000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f30380d3610 0x7f30380b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.415+0000 7f303f000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f303815de30 0x7f30381603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.415+0000 7f303f000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30380d0e60 con 0x7f303815de30 2024-03-08T06:06:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.415+0000 7f303f000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30380d2260 con 0x7f30380d0a00 2024-03-08T06:06:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.415+0000 7f303f000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30380d0fe0 con 0x7f30380d3610 2024-03-08T06:06:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.415+0000 7f303e600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f303815de30 0x7f30381603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.415+0000 7f303e600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f303815de30 0x7f30381603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39942/0 (socket says 172.21.15.40:39942) 2024-03-08T06:06:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.415+0000 7f303e600640 1 -- 172.21.15.40:0/3885039700 learned_addr learned my addr 172.21.15.40:0/3885039700 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.415+0000 7f303dc00640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380d0a00 0x7f30381f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.415+0000 7f303d200640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f30380d3610 0x7f30380b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f303e600640 1 -- 172.21.15.40:0/3885039700 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f30380d3610 msgr2=0x7f30380b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f303e600640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f30380d3610 0x7f30380b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f303e600640 1 -- 172.21.15.40:0/3885039700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380d0a00 msgr2=0x7f30381f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f303e600640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380d0a00 0x7f30381f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f303e600640 1 -- 172.21.15.40:0/3885039700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30381f7c30 con 0x7f303815de30 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f303d200640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f30380d3610 0x7f30380b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f303e600640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f303815de30 0x7f30381603a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f303000c780 tx=0x7f303000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f3035600640 1 -- 172.21.15.40:0/3885039700 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030004b60 con 0x7f303815de30 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f3035600640 1 -- 172.21.15.40:0/3885039700 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3030004d00 con 0x7f303815de30 2024-03-08T06:06:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f3035600640 1 -- 172.21.15.40:0/3885039700 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303000aa90 con 0x7f303815de30 2024-03-08T06:06:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f303f000640 1 -- 172.21.15.40:0/3885039700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30380ccec0 con 0x7f303815de30 2024-03-08T06:06:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.416+0000 7f303f000640 1 -- 172.21.15.40:0/3885039700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30381f3e70 con 0x7f303815de30 2024-03-08T06:06:24.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.418+0000 7f303f000640 1 -- 172.21.15.40:0/3885039700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3038075100 con 0x7f303815de30 2024-03-08T06:06:24.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.419+0000 7f3035600640 1 -- 172.21.15.40:0/3885039700 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f303000ac30 con 0x7f303815de30 2024-03-08T06:06:24.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.419+0000 7f3035600640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f301806b4d0 0x7f301806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:24.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.420+0000 7f3035600640 1 -- 172.21.15.40:0/3885039700 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f303008a560 con 0x7f303815de30 2024-03-08T06:06:24.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.420+0000 7f303dc00640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f301806b4d0 0x7f301806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:24.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.420+0000 7f303dc00640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f301806b4d0 0x7f301806d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f302c006fd0 tx=0x7f302c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.424+0000 7f3035600640 1 -- 172.21.15.40:0/3885039700 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3030020070 con 0x7f303815de30 2024-03-08T06:06:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:24.614+0000 7f303f000640 1 -- 172.21.15.40:0/3885039700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f30380b3a30 con 0x7f303815de30 2024-03-08T06:06:25.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.286+0000 7f3035600640 1 -- 172.21.15.40:0/3885039700 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7f30381f7c30 con 0x7f303815de30 2024-03-08T06:06:25.286 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-03-08T06:06:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.289+0000 7f303f000640 1 -- 172.21.15.40:0/3885039700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f301806b4d0 msgr2=0x7f301806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.289+0000 7f303f000640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f301806b4d0 0x7f301806d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f302c006fd0 tx=0x7f302c008040 comp rx=0 tx=0).stop 2024-03-08T06:06:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.289+0000 7f303f000640 1 -- 172.21.15.40:0/3885039700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f303815de30 msgr2=0x7f30381603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.290+0000 7f303f000640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f303815de30 0x7f30381603a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f303000c780 tx=0x7f303000cc40 comp rx=0 tx=0).stop 2024-03-08T06:06:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.290+0000 7f303f000640 1 -- 172.21.15.40:0/3885039700 shutdown_connections 2024-03-08T06:06:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.290+0000 7f303f000640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f301806b4d0 0x7f301806d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.290+0000 7f303f000640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f303815de30 0x7f30381603a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.290+0000 7f303f000640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f30380d3610 0x7f30380b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.290+0000 7f303f000640 1 --2- 172.21.15.40:0/3885039700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30380d0a00 0x7f30381f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.290+0000 7f303f000640 1 -- 172.21.15.40:0/3885039700 >> 172.21.15.40:0/3885039700 conn(0x7f303806c740 msgr2=0x7f30380b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:25.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.290+0000 7f303f000640 1 -- 172.21.15.40:0/3885039700 shutdown_connections 2024-03-08T06:06:25.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.290+0000 7f303f000640 1 -- 172.21.15.40:0/3885039700 wait complete. 2024-03-08T06:06:25.315 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-08T06:06:25.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.523+0000 7f566bc00640 1 Processor -- start 2024-03-08T06:06:25.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.524+0000 7f566bc00640 1 -- start start 2024-03-08T06:06:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.525+0000 7f566bc00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56641455b0 0x7f566415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.525+0000 7f566bc00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5664071950 0x7f5664071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.525+0000 7f566bc00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56640cd5a0 con 0x7f56641604f0 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.525+0000 7f566bc00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56640ccda0 con 0x7f56641455b0 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.525+0000 7f566bc00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56640cdc20 con 0x7f5664071950 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.526+0000 7f5669200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56641455b0 0x7f566415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.526+0000 7f5663e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f56641604f0 0x7f56640cc690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42066/0 (socket says 172.21.15.40:42066) 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.526+0000 7f5669200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56641455b0 0x7f566415c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39958/0 (socket says 172.21.15.40:39958) 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.526+0000 7f5663e00640 1 -- 172.21.15.40:0/3194285294 learned_addr learned my addr 172.21.15.40:0/3194285294 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.526+0000 7f5669c00640 1 --2- 172.21.15.40:0/3194285294 >> v2:172.21.15.53:3300/0 conn(0x7f5664071950 0x7f5664071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.526+0000 7f5669200640 1 -- 172.21.15.40:0/3194285294 >> v2:172.21.15.53:3300/0 conn(0x7f5664071950 msgr2=0x7f5664071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.526+0000 7f5669200640 1 --2- 172.21.15.40:0/3194285294 >> v2:172.21.15.53:3300/0 conn(0x7f5664071950 0x7f5664071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.526+0000 7f5669200640 1 -- 172.21.15.40:0/3194285294 >> v1:172.21.15.40:6789/0 conn(0x7f56641604f0 legacy=0x7f56640cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.526+0000 7f5669200640 1 -- 172.21.15.40:0/3194285294 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56640ccfe0 con 0x7f56641455b0 2024-03-08T06:06:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.526+0000 7f5663400640 1 -- 172.21.15.40:0/3194285294 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104813122 0 0) 0x7f56640cd5a0 con 0x7f56641604f0 2024-03-08T06:06:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.527+0000 7f5669200640 1 --2- 172.21.15.40:0/3194285294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56641455b0 0x7f566415c1d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f566406f0f0 tx=0x7f565802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f922cd74d338e6b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.528+0000 7f5663400640 1 -- 172.21.15.40:0/3194285294 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658035020 con 0x7f56641455b0 2024-03-08T06:06:25.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.528+0000 7f5663400640 1 -- 172.21.15.40:0/3194285294 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5658031830 con 0x7f56641455b0 2024-03-08T06:06:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.528+0000 7f5663400640 1 -- 172.21.15.40:0/3194285294 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658031c00 con 0x7f56641455b0 2024-03-08T06:06:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.528+0000 7f566bc00640 1 -- 172.21.15.40:0/3194285294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56641455b0 msgr2=0x7f566415c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.528+0000 7f566bc00640 1 --2- 172.21.15.40:0/3194285294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56641455b0 0x7f566415c1d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f566406f0f0 tx=0x7f565802b8d0 comp rx=0 tx=0).stop 2024-03-08T06:06:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.528+0000 7f566bc00640 1 -- 172.21.15.40:0/3194285294 shutdown_connections 2024-03-08T06:06:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.528+0000 7f566bc00640 1 --2- 172.21.15.40:0/3194285294 >> v2:172.21.15.53:3300/0 conn(0x7f5664071950 0x7f5664071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.528+0000 7f566bc00640 1 --2- 172.21.15.40:0/3194285294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56641455b0 0x7f566415c1d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.528+0000 7f566bc00640 1 -- 172.21.15.40:0/3194285294 >> 172.21.15.40:0/3194285294 conn(0x7f566406ae30 msgr2=0x7f566406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.528+0000 7f566bc00640 1 -- 172.21.15.40:0/3194285294 shutdown_connections 2024-03-08T06:06:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.529+0000 7f566bc00640 1 -- 172.21.15.40:0/3194285294 wait complete. 2024-03-08T06:06:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.530+0000 7f566bc00640 1 Processor -- start 2024-03-08T06:06:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.530+0000 7f566bc00640 1 -- start start 2024-03-08T06:06:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.531+0000 7f566bc00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5664071950 0x7f566413b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.531+0000 7f566bc00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56641455b0 0x7f56641f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.531+0000 7f566bc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56641604f0 0x7f566415b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.531+0000 7f566bc00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640cd8b0 con 0x7f5664071950 2024-03-08T06:06:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.531+0000 7f566bc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640722d0 con 0x7f56641604f0 2024-03-08T06:06:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.531+0000 7f566bc00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566415c710 con 0x7f56641455b0 2024-03-08T06:06:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.532+0000 7f5669200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5664071950 0x7f566413b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.532+0000 7f5669200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5664071950 0x7f566413b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39960/0 (socket says 172.21.15.40:39960) 2024-03-08T06:06:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.532+0000 7f5669200640 1 -- 172.21.15.40:0/953628545 learned_addr learned my addr 172.21.15.40:0/953628545 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.532+0000 7f5669c00640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56641604f0 0x7f566415b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.532+0000 7f5663e00640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56641455b0 0x7f56641f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.532+0000 7f5669200640 1 -- 172.21.15.40:0/953628545 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56641455b0 msgr2=0x7f56641f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.532+0000 7f5669200640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56641455b0 0x7f56641f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.532+0000 7f5669200640 1 -- 172.21.15.40:0/953628545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56641604f0 msgr2=0x7f566415b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.532+0000 7f5669200640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56641604f0 0x7f566415b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.532+0000 7f5669200640 1 -- 172.21.15.40:0/953628545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566413ba20 con 0x7f5664071950 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.533+0000 7f5669200640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5664071950 0x7f566413b1d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5658009890 tx=0x7f56580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.533+0000 7f5660c00640 1 -- 172.21.15.40:0/953628545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658041070 con 0x7f5664071950 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.533+0000 7f5660c00640 1 -- 172.21.15.40:0/953628545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565803a950 con 0x7f5664071950 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.533+0000 7f566bc00640 1 -- 172.21.15.40:0/953628545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566413bd00 con 0x7f5664071950 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.533+0000 7f5660c00640 1 -- 172.21.15.40:0/953628545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658035040 con 0x7f5664071950 2024-03-08T06:06:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.533+0000 7f566bc00640 1 -- 172.21.15.40:0/953628545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566413d950 con 0x7f5664071950 2024-03-08T06:06:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.534+0000 7f566bc00640 1 -- 172.21.15.40:0/953628545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566413d750 con 0x7f5664071950 2024-03-08T06:06:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.535+0000 7f5660c00640 1 -- 172.21.15.40:0/953628545 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5658038020 con 0x7f5664071950 2024-03-08T06:06:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.536+0000 7f5660c00640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:25.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.536+0000 7f5660c00640 1 -- 172.21.15.40:0/953628545 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f56580abc20 con 0x7f5664071950 2024-03-08T06:06:25.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.537+0000 7f5663e00640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:25.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.537+0000 7f5663e00640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f564406b4d0 0x7f564406d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f566413ebe0 tx=0x7f5650009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:25.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.541+0000 7f5660c00640 1 -- 172.21.15.40:0/953628545 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f565807ab60 con 0x7f5664071950 2024-03-08T06:06:25.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:25.734+0000 7f566bc00640 1 -- 172.21.15.40:0/953628545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5664067cb0 con 0x7f5664071950 2024-03-08T06:06:26.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.308+0000 7f5660c00640 1 -- 172.21.15.40:0/953628545 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7f565807a460 con 0x7f5664071950 2024-03-08T06:06:26.308 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-03-08T06:06:26.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.312+0000 7f566bc00640 1 -- 172.21.15.40:0/953628545 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f564406b4d0 msgr2=0x7f564406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:26.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.312+0000 7f566bc00640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f564406b4d0 0x7f564406d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f566413ebe0 tx=0x7f5650009040 comp rx=0 tx=0).stop 2024-03-08T06:06:26.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.312+0000 7f566bc00640 1 -- 172.21.15.40:0/953628545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5664071950 msgr2=0x7f566413b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:26.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.312+0000 7f566bc00640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5664071950 0x7f566413b1d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5658009890 tx=0x7f56580046c0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.312+0000 7f566bc00640 1 -- 172.21.15.40:0/953628545 shutdown_connections 2024-03-08T06:06:26.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.312+0000 7f566bc00640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.312+0000 7f566bc00640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56641604f0 0x7f566415b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.313+0000 7f566bc00640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56641455b0 0x7f56641f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.313+0000 7f566bc00640 1 --2- 172.21.15.40:0/953628545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5664071950 0x7f566413b1d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.313+0000 7f566bc00640 1 -- 172.21.15.40:0/953628545 >> 172.21.15.40:0/953628545 conn(0x7f566406ae30 msgr2=0x7f566415e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:26.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.313+0000 7f566bc00640 1 -- 172.21.15.40:0/953628545 shutdown_connections 2024-03-08T06:06:26.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.315+0000 7f566bc00640 1 -- 172.21.15.40:0/953628545 wait complete. 2024-03-08T06:06:26.341 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-08T06:06:26.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.556+0000 7faddf600640 1 Processor -- start 2024-03-08T06:06:26.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.557+0000 7faddf600640 1 -- start start 2024-03-08T06:06:26.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.558+0000 7faddf600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadd815ddb0 0x7fadd80ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:26.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.559+0000 7faddf600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fadd806ac50 0x7fadd81619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:26.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.559+0000 7faddf600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd80d0550 con 0x7fadd806fdb0 2024-03-08T06:06:26.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.559+0000 7faddf600640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd80d0850 con 0x7fadd806ac50 2024-03-08T06:06:26.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.559+0000 7faddf600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8162220 con 0x7fadd815ddb0 2024-03-08T06:06:26.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.559+0000 7faddd600640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fadd806fdb0 0x7fadd8070240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42070/0 (socket says 172.21.15.40:42070) 2024-03-08T06:06:26.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.559+0000 7faddd600640 1 -- 172.21.15.40:0/2481384628 learned_addr learned my addr 172.21.15.40:0/2481384628 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.559+0000 7fadd7e00640 1 --2- 172.21.15.40:0/2481384628 >> v2:172.21.15.53:3300/0 conn(0x7fadd806ac50 0x7fadd81619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.559+0000 7faddcc00640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadd815ddb0 0x7fadd80ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.559+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022502584 0 0) 0x7fadd80d0550 con 0x7fadd806fdb0 2024-03-08T06:06:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadbc003520 con 0x7fadd806fdb0 2024-03-08T06:06:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2463676679 0 0) 0x7fadbc003520 con 0x7fadd806fdb0 2024-03-08T06:06:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadd815ddb0 msgr2=0x7fadd80ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadd815ddb0 0x7fadd80ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 >> v2:172.21.15.53:3300/0 conn(0x7fadd806ac50 msgr2=0x7fadd81619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 --2- 172.21.15.40:0/2481384628 >> v2:172.21.15.53:3300/0 conn(0x7fadd806ac50 0x7fadd81619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd8162500 con 0x7fadd806fdb0 2024-03-08T06:06:26.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fadd0001670 con 0x7fadd806fdb0 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 >> v1:172.21.15.40:6789/0 conn(0x7fadd806fdb0 legacy=0x7fadd8070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadbc0039b0 0x7fadbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadbc004800 0x7fadbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadd815ddb0 0x7fadbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80d0850 con 0x7fadbc004800 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8162220 con 0x7fadbc0039b0 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc00b9c0 con 0x7fadd815ddb0 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.560+0000 7fadd7e00640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadbc004800 0x7fadbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.561+0000 7faddcc00640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadbc0039b0 0x7fadbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.561+0000 7faddd600640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadd815ddb0 0x7fadbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.561+0000 7fadd7e00640 1 -- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadd815ddb0 msgr2=0x7fadbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.561+0000 7fadd7e00640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadd815ddb0 0x7fadbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.561+0000 7fadd7e00640 1 -- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadbc0039b0 msgr2=0x7fadbc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:26.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.561+0000 7fadd7e00640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadbc0039b0 0x7fadbc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.561+0000 7fadd7e00640 1 -- 172.21.15.40:0/2481384628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc003520 con 0x7fadbc004800 2024-03-08T06:06:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.561+0000 7faddcc00640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadbc0039b0 0x7fadbc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:26.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.562+0000 7fadd7e00640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadbc004800 0x7fadbc004c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009fc0 tx=0x7fadbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e5bddcae92589e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.562+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0039070 con 0x7fadbc004800 2024-03-08T06:06:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.562+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc002cc40 con 0x7fadbc004800 2024-03-08T06:06:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.562+0000 7fadd7400640 1 -- 172.21.15.40:0/2481384628 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0009c10 con 0x7fadbc004800 2024-03-08T06:06:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.563+0000 7faddf600640 1 -- 172.21.15.40:0/2481384628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadbc004800 msgr2=0x7fadbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.563+0000 7faddf600640 1 --2- 172.21.15.40:0/2481384628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadbc004800 0x7fadbc004c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009fc0 tx=0x7fadbc005830 comp rx=0 tx=0).stop 2024-03-08T06:06:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.563+0000 7faddd600640 1 -- 172.21.15.40:0/2481384628 reap_dead start 2024-03-08T06:06:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.563+0000 7faddf600640 1 -- 172.21.15.40:0/2481384628 shutdown_connections 2024-03-08T06:06:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.563+0000 7faddf600640 1 -- 172.21.15.40:0/2481384628 >> 172.21.15.40:0/2481384628 conn(0x7fadd8099a70 msgr2=0x7fadd8099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.563+0000 7faddf600640 1 -- 172.21.15.40:0/2481384628 shutdown_connections 2024-03-08T06:06:26.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.563+0000 7faddf600640 1 -- 172.21.15.40:0/2481384628 wait complete. 2024-03-08T06:06:26.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.565+0000 7faddf600640 1 Processor -- start 2024-03-08T06:06:26.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.565+0000 7faddf600640 1 -- start start 2024-03-08T06:06:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.566+0000 7faddf600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadbc004800 0x7fadd814e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.566+0000 7faddf600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadd80ceef0 0x7fadd80cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.566+0000 7faddf600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd8160500 0x7fadd8160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.566+0000 7faddf600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd815f1a0 con 0x7fadd8160500 2024-03-08T06:06:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.566+0000 7faddf600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81627e0 con 0x7fadd80ceef0 2024-03-08T06:06:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.566+0000 7faddf600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80ce640 con 0x7fadbc004800 2024-03-08T06:06:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.566+0000 7faddd600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd8160500 0x7fadd8160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.566+0000 7faddd600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd8160500 0x7fadd8160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39976/0 (socket says 172.21.15.40:39976) 2024-03-08T06:06:26.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.566+0000 7faddd600640 1 -- 172.21.15.40:0/1046280980 learned_addr learned my addr 172.21.15.40:0/1046280980 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.567+0000 7faddd600640 1 -- 172.21.15.40:0/1046280980 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadbc004800 msgr2=0x7fadd814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.567+0000 7faddd600640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadbc004800 0x7fadd814e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.567+0000 7faddd600640 1 -- 172.21.15.40:0/1046280980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadd80ceef0 msgr2=0x7fadd80cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.567+0000 7faddd600640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadd80ceef0 0x7fadd80cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.567+0000 7faddd600640 1 -- 172.21.15.40:0/1046280980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd81612a0 con 0x7fadd8160500 2024-03-08T06:06:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.567+0000 7faddd600640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd8160500 0x7fadd8160980 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fadd80711e0 tx=0x7fadd000e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.567+0000 7fadd4c00640 1 -- 172.21.15.40:0/1046280980 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0019070 con 0x7fadd8160500 2024-03-08T06:06:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.567+0000 7fadd4c00640 1 -- 172.21.15.40:0/1046280980 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd0010510 con 0x7fadd8160500 2024-03-08T06:06:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.567+0000 7fadd4c00640 1 -- 172.21.15.40:0/1046280980 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0010900 con 0x7fadd8160500 2024-03-08T06:06:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.567+0000 7faddf600640 1 -- 172.21.15.40:0/1046280980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd814eec0 con 0x7fadd8160500 2024-03-08T06:06:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.567+0000 7faddf600640 1 -- 172.21.15.40:0/1046280980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd80c8900 con 0x7fadd8160500 2024-03-08T06:06:26.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.569+0000 7faddf600640 1 -- 172.21.15.40:0/1046280980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd8067cb0 con 0x7fadd8160500 2024-03-08T06:06:26.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.569+0000 7fadd4c00640 1 -- 172.21.15.40:0/1046280980 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadd001c070 con 0x7fadd8160500 2024-03-08T06:06:26.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.570+0000 7fadd4c00640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fadb406b4d0 0x7fadb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:26.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.570+0000 7fadd4c00640 1 -- 172.21.15.40:0/1046280980 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fadd0089480 con 0x7fadd8160500 2024-03-08T06:06:26.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.571+0000 7faddcc00640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fadb406b4d0 0x7fadb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:26.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.571+0000 7faddcc00640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fadb406b4d0 0x7fadb406d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009930 tx=0x7fadc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.575+0000 7fadd4c00640 1 -- 172.21.15.40:0/1046280980 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fadd0058560 con 0x7fadd8160500 2024-03-08T06:06:26.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.779+0000 7faddf600640 1 -- 172.21.15.40:0/1046280980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fadd80cbf30 con 0x7fadd8160500 2024-03-08T06:06:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.785+0000 7fadd4c00640 1 -- 172.21.15.40:0/1046280980 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fadd0057e60 con 0x7fadd8160500 2024-03-08T06:06:26.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.787+0000 7faddf600640 1 -- 172.21.15.40:0/1046280980 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fadb406b4d0 msgr2=0x7fadb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:26.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.788+0000 7faddf600640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fadb406b4d0 0x7fadb406d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009930 tx=0x7fadc8008040 comp rx=0 tx=0).stop 2024-03-08T06:06:26.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.788+0000 7faddf600640 1 -- 172.21.15.40:0/1046280980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd8160500 msgr2=0x7fadd8160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:26.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.788+0000 7faddf600640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd8160500 0x7fadd8160980 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fadd80711e0 tx=0x7fadd000e960 comp rx=0 tx=0).stop 2024-03-08T06:06:26.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.788+0000 7faddf600640 1 -- 172.21.15.40:0/1046280980 shutdown_connections 2024-03-08T06:06:26.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.788+0000 7faddf600640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fadb406b4d0 0x7fadb406d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.788+0000 7faddf600640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd8160500 0x7fadd8160980 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.788+0000 7faddf600640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadd80ceef0 0x7fadd80cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.788+0000 7faddf600640 1 --2- 172.21.15.40:0/1046280980 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fadbc004800 0x7fadd814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:26.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.788+0000 7faddf600640 1 -- 172.21.15.40:0/1046280980 >> 172.21.15.40:0/1046280980 conn(0x7fadd8099a70 msgr2=0x7fadd80c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:26.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.788+0000 7faddf600640 1 -- 172.21.15.40:0/1046280980 shutdown_connections 2024-03-08T06:06:26.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:26.788+0000 7faddf600640 1 -- 172.21.15.40:0/1046280980 wait complete. 2024-03-08T06:06:26.809 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters' 2024-03-08T06:06:27.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.017+0000 7f21ef200640 1 Processor -- start 2024-03-08T06:06:27.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.018+0000 7f21ef200640 1 -- start start 2024-03-08T06:06:27.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.019+0000 7f21ef200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21e8161740 0x7f21e80b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:27.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.019+0000 7f21ef200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f21e80716e0 0x7f21e8071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:27.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.019+0000 7f21ef200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e8161d20 con 0x7f21e815b6c0 2024-03-08T06:06:27.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.019+0000 7f21ef200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e815e280 con 0x7f21e8161740 2024-03-08T06:06:27.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.019+0000 7f21ef200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e80b4740 con 0x7f21e80716e0 2024-03-08T06:06:27.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.020+0000 7f21ede00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f21e815b6c0 0x7f21e80adec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36404/0 (socket says 172.21.15.40:36404) 2024-03-08T06:06:27.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.020+0000 7f21ede00640 1 -- 172.21.15.40:0/3761033934 learned_addr learned my addr 172.21.15.40:0/3761033934 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:27.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.020+0000 7f21ed400640 1 --2- 172.21.15.40:0/3761033934 >> v2:172.21.15.40:3300/0 conn(0x7f21e8161740 0x7f21e80b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.020+0000 7f21eca00640 1 -- 172.21.15.40:0/3761033934 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798044285 0 0) 0x7f21e8161d20 con 0x7f21e815b6c0 2024-03-08T06:06:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.020+0000 7f21eca00640 1 -- 172.21.15.40:0/3761033934 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21d0003520 con 0x7f21e815b6c0 2024-03-08T06:06:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.020+0000 7f21ee800640 1 --2- 172.21.15.40:0/3761033934 >> v2:172.21.15.53:3300/0 conn(0x7f21e80716e0 0x7f21e8071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.020+0000 7f21ed400640 1 -- 172.21.15.40:0/3761033934 >> v2:172.21.15.53:3300/0 conn(0x7f21e80716e0 msgr2=0x7f21e8071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.020+0000 7f21ed400640 1 --2- 172.21.15.40:0/3761033934 >> v2:172.21.15.53:3300/0 conn(0x7f21e80716e0 0x7f21e8071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.020+0000 7f21ed400640 1 -- 172.21.15.40:0/3761033934 >> v1:172.21.15.40:6789/0 conn(0x7f21e815b6c0 legacy=0x7f21e80adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:27.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.020+0000 7f21ed400640 1 -- 172.21.15.40:0/3761033934 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8162000 con 0x7f21e8161740 2024-03-08T06:06:27.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.020+0000 7f21ee800640 1 --2- 172.21.15.40:0/3761033934 >> v2:172.21.15.53:3300/0 conn(0x7f21e80716e0 0x7f21e8071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.022+0000 7f21ed400640 1 --2- 172.21.15.40:0/3761033934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8161740 0x7f21e80b49e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f21e8072600 tx=0x7f21d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47b6abfb34f8fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.022+0000 7f21eca00640 1 -- 172.21.15.40:0/3761033934 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8035020 con 0x7f21e8161740 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.022+0000 7f21eca00640 1 -- 172.21.15.40:0/3761033934 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d8031830 con 0x7f21e8161740 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.022+0000 7f21eca00640 1 -- 172.21.15.40:0/3761033934 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8031c00 con 0x7f21e8161740 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.022+0000 7f21ef200640 1 -- 172.21.15.40:0/3761033934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8161740 msgr2=0x7f21e80b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.022+0000 7f21ef200640 1 --2- 172.21.15.40:0/3761033934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8161740 0x7f21e80b49e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f21e8072600 tx=0x7f21d802b8d0 comp rx=0 tx=0).stop 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.023+0000 7f21ef200640 1 -- 172.21.15.40:0/3761033934 shutdown_connections 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.023+0000 7f21ef200640 1 --2- 172.21.15.40:0/3761033934 >> v2:172.21.15.53:3300/0 conn(0x7f21e80716e0 0x7f21e8071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.023+0000 7f21ef200640 1 --2- 172.21.15.40:0/3761033934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8161740 0x7f21e80b49e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.023+0000 7f21ef200640 1 -- 172.21.15.40:0/3761033934 >> 172.21.15.40:0/3761033934 conn(0x7f21e80b1850 msgr2=0x7f21e80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.023+0000 7f21ef200640 1 -- 172.21.15.40:0/3761033934 shutdown_connections 2024-03-08T06:06:27.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.023+0000 7f21ef200640 1 -- 172.21.15.40:0/3761033934 wait complete. 2024-03-08T06:06:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.024+0000 7f21ef200640 1 Processor -- start 2024-03-08T06:06:27.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.025+0000 7f21ef200640 1 -- start start 2024-03-08T06:06:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ef200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e80716e0 0x7f21e815a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ef200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21e815b6c0 0x7f21e80c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ef200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8161740 0x7f21e80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ef200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e815baa0 con 0x7f21e8161740 2024-03-08T06:06:27.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ef200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e81625c0 con 0x7f21e80716e0 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ef200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80ae5d0 con 0x7f21e815b6c0 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ee800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8161740 0x7f21e80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ee800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8161740 0x7f21e80ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36174/0 (socket says 172.21.15.40:36174) 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ee800640 1 -- 172.21.15.40:0/423668628 learned_addr learned my addr 172.21.15.40:0/423668628 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ede00640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e80716e0 0x7f21e815a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ed400640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21e815b6c0 0x7f21e80c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ee800640 1 -- 172.21.15.40:0/423668628 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21e815b6c0 msgr2=0x7f21e80c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ee800640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21e815b6c0 0x7f21e80c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ee800640 1 -- 172.21.15.40:0/423668628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e80716e0 msgr2=0x7f21e815a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ee800640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e80716e0 0x7f21e815a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.026+0000 7f21ee800640 1 -- 172.21.15.40:0/423668628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80c4a80 con 0x7f21e8161740 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.027+0000 7f21ee800640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8161740 0x7f21e80ce420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f21e400cef0 tx=0x7f21e400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.027+0000 7f21e2000640 1 -- 172.21.15.40:0/423668628 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e401a070 con 0x7f21e8161740 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.027+0000 7f21e2000640 1 -- 172.21.15.40:0/423668628 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e400d860 con 0x7f21e8161740 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.027+0000 7f21ef200640 1 -- 172.21.15.40:0/423668628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e80c4d60 con 0x7f21e8161740 2024-03-08T06:06:27.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.027+0000 7f21e2000640 1 -- 172.21.15.40:0/423668628 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e400dc50 con 0x7f21e8161740 2024-03-08T06:06:27.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.027+0000 7f21ef200640 1 -- 172.21.15.40:0/423668628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e80c5320 con 0x7f21e8161740 2024-03-08T06:06:27.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.028+0000 7f21ef200640 1 -- 172.21.15.40:0/423668628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e8075100 con 0x7f21e8161740 2024-03-08T06:06:27.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.029+0000 7f21e2000640 1 -- 172.21.15.40:0/423668628 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21e401d070 con 0x7f21e8161740 2024-03-08T06:06:27.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.030+0000 7f21e2000640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f21cc06b4d0 0x7f21cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:27.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.030+0000 7f21e2000640 1 -- 172.21.15.40:0/423668628 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f21e408a790 con 0x7f21e8161740 2024-03-08T06:06:27.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.030+0000 7f21ede00640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f21cc06b4d0 0x7f21cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:27.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.031+0000 7f21ede00640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f21cc06b4d0 0x7f21cc06d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005880 tx=0x7f21dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:27.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.036+0000 7f21e2000640 1 -- 172.21.15.40:0/423668628 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21e4059910 con 0x7f21e8161740 2024-03-08T06:06:27.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:27.313+0000 7f21ef200640 1 -- 172.21.15.40:0/423668628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]} v 0) v1 -- 0x7f21e80aeef0 con 0x7f21e8161740 2024-03-08T06:06:28.248 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:06:28.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:06:28.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.318+0000 7f21e2000640 1 -- 172.21.15.40:0/423668628 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]}]=0 v135) v1 ==== 164+0+0 (secure 0 0 0) 0x7f21e4059210 con 0x7f21e8161740 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 -- 172.21.15.40:0/423668628 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f21cc06b4d0 msgr2=0x7f21cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f21cc06b4d0 0x7f21cc06d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005880 tx=0x7f21dc009040 comp rx=0 tx=0).stop 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 -- 172.21.15.40:0/423668628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8161740 msgr2=0x7f21e80ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8161740 0x7f21e80ce420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f21e400cef0 tx=0x7f21e400a710 comp rx=0 tx=0).stop 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 -- 172.21.15.40:0/423668628 shutdown_connections 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f21cc06b4d0 0x7f21cc06d990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8161740 0x7f21e80ce420 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f21e815b6c0 0x7f21e80c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 --2- 172.21.15.40:0/423668628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21e80716e0 0x7f21e815a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 -- 172.21.15.40:0/423668628 >> 172.21.15.40:0/423668628 conn(0x7f21e80b1850 msgr2=0x7f21e80af650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 -- 172.21.15.40:0/423668628 shutdown_connections 2024-03-08T06:06:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:28.322+0000 7f21ef200640 1 -- 172.21.15.40:0/423668628 wait complete. 2024-03-08T06:06:28.343 INFO:tasks.cephfs_test_runner:That the per-client labelled perf counters depict the clients ... ERROR 2024-03-08T06:06:28.343 INFO:tasks.cephfs_test_runner: 2024-03-08T06:06:28.343 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-08T06:06:28.343 INFO:tasks.cephfs_test_runner:ERROR: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T06:06:28.343 INFO:tasks.cephfs_test_runner:That the per-client labelled perf counters depict the clients 2024-03-08T06:06:28.343 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa/tasks/cephfs/test_admin.py", line 57, in test_per_client_labeled_perf_counters 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner: while proceed(): 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: 'wait for counters - client.4649' reached maximum tries (31) after waiting for 30 seconds 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner: 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner:Ran 1 test in 108.861s 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner: 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner: 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner:ERROR: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner:That the per-client labelled perf counters depict the clients 2024-03-08T06:06:28.344 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-08T06:06:28.345 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-08T06:06:28.345 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa/tasks/cephfs/test_admin.py", line 57, in test_per_client_labeled_perf_counters 2024-03-08T06:06:28.345 INFO:tasks.cephfs_test_runner: while proceed(): 2024-03-08T06:06:28.345 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ 2024-03-08T06:06:28.345 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2024-03-08T06:06:28.345 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: 'wait for counters - client.4649' reached maximum tries (31) after waiting for 30 seconds 2024-03-08T06:06:28.345 INFO:tasks.cephfs_test_runner: 2024-03-08T06:06:28.345 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T06:06:28.565 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b4a7ad8048164354b4cde0de50a361ac Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_joscollin_ceph_de7a7e179fbee9e314fa702f8f6c09d121c5e1a7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T06:06:28.566 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-08T06:06:28.575 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-08T06:06:28.624 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-08T06:06:28.624 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:06:28.624 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:06:28.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054202,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-08T06:06:28.728 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-03-08T06:06:28.728 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:06:28.784 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054202,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-08T06:06:28.785 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:06:28.785 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:06:28.845 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054202,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-08T06:06:28.845 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-03-08T06:06:28.845 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T06:06:28.902 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054202,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-08T06:06:28.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:28.903 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-03-08T06:06:28.963 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-08T06:06:28.963 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-08T06:06:28.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:28.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T06:06:29.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:29.062 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-08T06:06:29.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:29.158 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-03-08T06:06:29.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:29.310 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-03-08T06:06:29.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:29.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-03-08T06:06:29.443 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-03-08T06:06:29.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:06:29.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:29.447 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-03-08T06:06:29.507 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-08T06:06:29.507 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-08T06:06:29.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:29.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T06:06:29.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:29.613 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-08T06:06:29.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:29.707 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-03-08T06:06:29.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:29.853 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-03-08T06:06:29.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T06:06:29.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-03-08T06:06:30.046 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2024-03-08T06:06:30.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:06:30.050 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-08T06:06:30.060 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-08T06:06:30.060 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:06:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.292+0000 7ff3e0a00640 1 Processor -- start 2024-03-08T06:06:30.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.293+0000 7ff3e0a00640 1 -- start start 2024-03-08T06:06:30.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.294+0000 7ff3e0a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff3dc145520 0x7ff3dc15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.294+0000 7ff3e0a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3dc15bea0 0x7ff3dc0716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.294+0000 7ff3e0a00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc15ceb0 con 0x7ff3dc0af040 2024-03-08T06:06:30.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.294+0000 7ff3e0a00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc15c330 con 0x7ff3dc145520 2024-03-08T06:06:30.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.294+0000 7ff3e0a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc15c610 con 0x7ff3dc15bea0 2024-03-08T06:06:30.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.295+0000 7ff3db200640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff3dc0af040 0x7ff3dc06d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36418/0 (socket says 172.21.15.40:36418) 2024-03-08T06:06:30.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.295+0000 7ff3db200640 1 -- 172.21.15.40:0/3434471713 learned_addr learned my addr 172.21.15.40:0/3434471713 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:30.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.295+0000 7ff3da800640 1 --2- 172.21.15.40:0/3434471713 >> v2:172.21.15.53:3300/0 conn(0x7ff3dc145520 0x7ff3dc15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.295+0000 7ff3d1e00640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3dc15bea0 0x7ff3dc0716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.295+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744674783 0 0) 0x7ff3dc15ceb0 con 0x7ff3dc0af040 2024-03-08T06:06:30.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.295+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c0003520 con 0x7ff3dc0af040 2024-03-08T06:06:30.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1637727483 0 0) 0x7ff3c0003520 con 0x7ff3dc0af040 2024-03-08T06:06:30.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3dc15bea0 msgr2=0x7ff3dc0716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:30.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3dc15bea0 0x7ff3dc0716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 >> v2:172.21.15.53:3300/0 conn(0x7ff3dc145520 msgr2=0x7ff3dc15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:30.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 --2- 172.21.15.40:0/3434471713 >> v2:172.21.15.53:3300/0 conn(0x7ff3dc145520 0x7ff3dc15d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc15c8f0 con 0x7ff3dc0af040 2024-03-08T06:06:30.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7ff3d40019f0 con 0x7ff3dc0af040 2024-03-08T06:06:30.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 >> v1:172.21.15.40:6789/0 conn(0x7ff3dc0af040 legacy=0x7ff3dc06d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3c00039b0 0x7ff3c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3c0004800 0x7ff3c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0005180 0x7ff3c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc15c330 con 0x7ff3c0005180 2024-03-08T06:06:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc15c610 con 0x7ff3c00039b0 2024-03-08T06:06:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c000bf00 con 0x7ff3c0004800 2024-03-08T06:06:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.296+0000 7ff3db200640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0005180 0x7ff3c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.297+0000 7ff3da800640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3c00039b0 0x7ff3c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.297+0000 7ff3d1e00640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3c0004800 0x7ff3c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.297+0000 7ff3db200640 1 -- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3c0004800 msgr2=0x7ff3c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.297+0000 7ff3db200640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3c0004800 0x7ff3c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.297+0000 7ff3db200640 1 -- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3c00039b0 msgr2=0x7ff3c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:30.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.297+0000 7ff3db200640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3c00039b0 0x7ff3c0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.297+0000 7ff3db200640 1 -- 172.21.15.40:0/3434471713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c0003520 con 0x7ff3c0005180 2024-03-08T06:06:30.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.297+0000 7ff3d1e00640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3c0004800 0x7ff3c0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:30.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.298+0000 7ff3db200640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0005180 0x7ff3c000b9c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400a7b0 tx=0x7ff3c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fadfcc390ac24d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:30.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.298+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d400a3d0 con 0x7ff3c0005180 2024-03-08T06:06:30.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.298+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d4034630 con 0x7ff3c0005180 2024-03-08T06:06:30.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.298+0000 7ff3d9e00640 1 -- 172.21.15.40:0/3434471713 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4034a60 con 0x7ff3c0005180 2024-03-08T06:06:30.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.299+0000 7ff3e0a00640 1 -- 172.21.15.40:0/3434471713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0005180 msgr2=0x7ff3c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.299+0000 7ff3e0a00640 1 --2- 172.21.15.40:0/3434471713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0005180 0x7ff3c000b9c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400a7b0 tx=0x7ff3c0005830 comp rx=0 tx=0).stop 2024-03-08T06:06:30.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.299+0000 7ff3db200640 1 -- 172.21.15.40:0/3434471713 reap_dead start 2024-03-08T06:06:30.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.299+0000 7ff3e0a00640 1 -- 172.21.15.40:0/3434471713 shutdown_connections 2024-03-08T06:06:30.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.299+0000 7ff3e0a00640 1 -- 172.21.15.40:0/3434471713 >> 172.21.15.40:0/3434471713 conn(0x7ff3dc0b1850 msgr2=0x7ff3dc0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:30.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.299+0000 7ff3e0a00640 1 -- 172.21.15.40:0/3434471713 shutdown_connections 2024-03-08T06:06:30.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.299+0000 7ff3e0a00640 1 -- 172.21.15.40:0/3434471713 wait complete. 2024-03-08T06:06:30.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.301+0000 7ff3e0a00640 1 Processor -- start 2024-03-08T06:06:30.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.301+0000 7ff3e0a00640 1 -- start start 2024-03-08T06:06:30.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.302+0000 7ff3e0a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3c0005180 0x7ff3dc13ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.302+0000 7ff3e0a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3dc154f70 0x7ff3dc1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.302+0000 7ff3e0a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3dc1553f0 0x7ff3dc1f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.302+0000 7ff3e0a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc06e590 con 0x7ff3dc154f70 2024-03-08T06:06:30.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.302+0000 7ff3e0a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc06e410 con 0x7ff3c0005180 2024-03-08T06:06:30.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.302+0000 7ff3e0a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc06e710 con 0x7ff3dc1553f0 2024-03-08T06:06:30.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.302+0000 7ff3d1e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3dc154f70 0x7ff3dc1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.303+0000 7ff3d1e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3dc154f70 0x7ff3dc1f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36182/0 (socket says 172.21.15.40:36182) 2024-03-08T06:06:30.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.303+0000 7ff3d1e00640 1 -- 172.21.15.40:0/2537053367 learned_addr learned my addr 172.21.15.40:0/2537053367 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.303+0000 7ff3da800640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3c0005180 0x7ff3dc13ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.303+0000 7ff3db200640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3dc1553f0 0x7ff3dc1f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.303+0000 7ff3d1e00640 1 -- 172.21.15.40:0/2537053367 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3dc1553f0 msgr2=0x7ff3dc1f3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.303+0000 7ff3d1e00640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3dc1553f0 0x7ff3dc1f3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.303+0000 7ff3d1e00640 1 -- 172.21.15.40:0/2537053367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3c0005180 msgr2=0x7ff3dc13ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.303+0000 7ff3d1e00640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3c0005180 0x7ff3dc13ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.303+0000 7ff3d1e00640 1 -- 172.21.15.40:0/2537053367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc152bd0 con 0x7ff3dc154f70 2024-03-08T06:06:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.303+0000 7ff3d1e00640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3dc154f70 0x7ff3dc1f3790 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4010910 tx=0x7ff3c4010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:30.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.304+0000 7ff3d3400640 1 -- 172.21.15.40:0/2537053367 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c400f040 con 0x7ff3dc154f70 2024-03-08T06:06:30.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.304+0000 7ff3d3400640 1 -- 172.21.15.40:0/2537053367 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c400ec20 con 0x7ff3dc154f70 2024-03-08T06:06:30.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.304+0000 7ff3e0a00640 1 -- 172.21.15.40:0/2537053367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3dc152eb0 con 0x7ff3dc154f70 2024-03-08T06:06:30.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.304+0000 7ff3d3400640 1 -- 172.21.15.40:0/2537053367 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c401c400 con 0x7ff3dc154f70 2024-03-08T06:06:30.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.304+0000 7ff3e0a00640 1 -- 172.21.15.40:0/2537053367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3dc153470 con 0x7ff3dc154f70 2024-03-08T06:06:30.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.305+0000 7ff3e0a00640 1 -- 172.21.15.40:0/2537053367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3dc075100 con 0x7ff3dc154f70 2024-03-08T06:06:30.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.306+0000 7ff3d3400640 1 -- 172.21.15.40:0/2537053367 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3c401c5a0 con 0x7ff3dc154f70 2024-03-08T06:06:30.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.307+0000 7ff3d3400640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff3b806b500 0x7ff3b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.307+0000 7ff3d3400640 1 -- 172.21.15.40:0/2537053367 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff3c408a410 con 0x7ff3dc154f70 2024-03-08T06:06:30.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.307+0000 7ff3da800640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff3b806b500 0x7ff3b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.308+0000 7ff3da800640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff3b806b500 0x7ff3b806d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009ac0 tx=0x7ff3cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:30.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.312+0000 7ff3d3400640 1 -- 172.21.15.40:0/2537053367 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3c4018030 con 0x7ff3dc154f70 2024-03-08T06:06:30.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.507+0000 7ff3e0a00640 1 -- 172.21.15.40:0/2537053367 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3dc06ee70 con 0x7ff3b806b500 2024-03-08T06:06:30.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.509+0000 7ff3d3400640 1 -- 172.21.15.40:0/2537053367 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13521 (secure 0 0 0) 0x7ff3dc06ee70 con 0x7ff3b806b500 2024-03-08T06:06:30.509 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:30.510 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-03-08T06:06:30.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.511+0000 7ff3e0a00640 1 -- 172.21.15.40:0/2537053367 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff3b806b500 msgr2=0x7ff3b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.511+0000 7ff3e0a00640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff3b806b500 0x7ff3b806d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009ac0 tx=0x7ff3cc008040 comp rx=0 tx=0).stop 2024-03-08T06:06:30.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.511+0000 7ff3e0a00640 1 -- 172.21.15.40:0/2537053367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3dc154f70 msgr2=0x7ff3dc1f3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.511+0000 7ff3e0a00640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3dc154f70 0x7ff3dc1f3790 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4010910 tx=0x7ff3c4010dd0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.511+0000 7ff3e0a00640 1 -- 172.21.15.40:0/2537053367 shutdown_connections 2024-03-08T06:06:30.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.511+0000 7ff3e0a00640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff3b806b500 0x7ff3b806d9c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.511+0000 7ff3e0a00640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff3dc1553f0 0x7ff3dc1f3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.512+0000 7ff3e0a00640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3dc154f70 0x7ff3dc1f3790 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.512+0000 7ff3e0a00640 1 --2- 172.21.15.40:0/2537053367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3c0005180 0x7ff3dc13ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.512+0000 7ff3e0a00640 1 -- 172.21.15.40:0/2537053367 >> 172.21.15.40:0/2537053367 conn(0x7ff3dc0b1850 msgr2=0x7ff3dc0afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:30.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.512+0000 7ff3e0a00640 1 -- 172.21.15.40:0/2537053367 shutdown_connections 2024-03-08T06:06:30.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.512+0000 7ff3e0a00640 1 -- 172.21.15.40:0/2537053367 wait complete. 2024-03-08T06:06:30.533 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-03-08T06:06:29.536443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279280,"kb_used_data":25684,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754695440,"statfs":{"total":773094113280,"available":772808130560,"internally_reserved":0,"allocated":26300416,"data_stored":16134347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6850374,"num_objects":-1023,"num_object_clones":0,"num_object_copies":-2046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2084,"num_write_kb":-8678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.382625"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":196,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-08T06:06:23.761334+0000","last_change":"2024-03-08T06:03:57.473597+0000","last_active":"2024-03-08T06:06:23.761334+0000","last_peered":"2024-03-08T06:06:23.761334+0000","last_clean":"2024-03-08T06:06:23.761334+0000","last_became_active":"2024-03-08T06:03:57.468593+0000","last_became_peered":"2024-03-08T06:03:57.468593+0000","last_unstale":"2024-03-08T06:06:23.761334+0000","last_undegraded":"2024-03-08T06:06:23.761334+0000","last_fullsized":"2024-03-08T06:06:23.761334+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_clean_scrub_stamp":"2024-03-08T06:03:56.214441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:45:28.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2035712,"data_stored":748556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1077248,"data_stored":152382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":158764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35344,"kb_used_data":4164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336496,"statfs":{"total":96636764160,"available":96600571904,"internally_reserved":0,"allocated":4263936,"data_stored":2810042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43748,"kb_used_data":8408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328092,"statfs":{"total":96636764160,"available":96591966208,"internally_reserved":0,"allocated":8609792,"data_stored":7005029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":158038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41100,"kb_used_data":5760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330740,"statfs":{"total":96636764160,"available":96594677760,"internally_reserved":0,"allocated":5898240,"data_stored":4353282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1724416,"data_stored":748254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:06:30.533 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:06:30.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.762+0000 7f553ec00640 1 Processor -- start 2024-03-08T06:06:30.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.762+0000 7f553ec00640 1 -- start start 2024-03-08T06:06:30.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.764+0000 7f553ec00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5538161410 0x7f55380b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.764+0000 7f553ec00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f55380b6c90 0x7f55380d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.764+0000 7f553ec00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55381617f0 con 0x7f55380d3500 2024-03-08T06:06:30.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.764+0000 7f553ec00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55380d1060 con 0x7f55380b6c90 2024-03-08T06:06:30.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.764+0000 7f553ec00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55380d12b0 con 0x7f5538161410 2024-03-08T06:06:30.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.765+0000 7f553ce00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f55380d3500 0x7f55380b5570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36424/0 (socket says 172.21.15.40:36424) 2024-03-08T06:06:30.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.765+0000 7f553ce00640 1 -- 172.21.15.40:0/2503829969 learned_addr learned my addr 172.21.15.40:0/2503829969 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:30.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.765+0000 7f553d800640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5538161410 0x7f55380b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.765+0000 7f553e200640 1 --2- 172.21.15.40:0/2503829969 >> v2:172.21.15.53:3300/0 conn(0x7f55380b6c90 0x7f55380d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.765+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792368322 0 0) 0x7f55381617f0 con 0x7f55380d3500 2024-03-08T06:06:30.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.765+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5520003520 con 0x7f55380d3500 2024-03-08T06:06:30.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3993667512 0 0) 0x7f5520003520 con 0x7f55380d3500 2024-03-08T06:06:30.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5538161410 msgr2=0x7f55380b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:30.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5538161410 0x7f55380b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 >> v2:172.21.15.53:3300/0 conn(0x7f55380b6c90 msgr2=0x7f55380d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:30.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 --2- 172.21.15.40:0/2503829969 >> v2:172.21.15.53:3300/0 conn(0x7f55380b6c90 0x7f55380d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55380cebd0 con 0x7f55380d3500 2024-03-08T06:06:30.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f5524001670 con 0x7f55380d3500 2024-03-08T06:06:30.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 >> v1:172.21.15.40:6789/0 conn(0x7f55380d3500 legacy=0x7f55380b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5538161410 0x7f55200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 0x7f5520004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5520004d40 0x7f55200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380d1060 con 0x7f5520003f80 2024-03-08T06:06:30.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380d12b0 con 0x7f5520004d40 2024-03-08T06:06:30.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552000b9c0 con 0x7f5538161410 2024-03-08T06:06:30.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f553ce00640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 0x7f5520004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.766+0000 7f553d800640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5538161410 0x7f55200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.767+0000 7f553e200640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5520004d40 0x7f55200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.767+0000 7f553ce00640 1 -- 172.21.15.40:0/2503829969 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5538161410 msgr2=0x7f55200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.767+0000 7f553ce00640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5538161410 0x7f55200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.767+0000 7f553ce00640 1 -- 172.21.15.40:0/2503829969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5520004d40 msgr2=0x7f55200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.767+0000 7f553ce00640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5520004d40 0x7f55200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.767+0000 7f553ce00640 1 -- 172.21.15.40:0/2503829969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5520003520 con 0x7f5520003f80 2024-03-08T06:06:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.768+0000 7f553ce00640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 0x7f5520004800 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f552400c190 tx=0x7f5520005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60280dd1caa7ddef server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:30.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.768+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524007c80 con 0x7f5520003f80 2024-03-08T06:06:30.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.768+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5524030070 con 0x7f5520003f80 2024-03-08T06:06:30.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.768+0000 7f5537e00640 1 -- 172.21.15.40:0/2503829969 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524005d70 con 0x7f5520003f80 2024-03-08T06:06:30.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.769+0000 7f553ec00640 1 -- 172.21.15.40:0/2503829969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 msgr2=0x7f5520004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.769+0000 7f553ec00640 1 --2- 172.21.15.40:0/2503829969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 0x7f5520004800 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f552400c190 tx=0x7f5520005830 comp rx=0 tx=0).stop 2024-03-08T06:06:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.769+0000 7f553e200640 1 -- 172.21.15.40:0/2503829969 reap_dead start 2024-03-08T06:06:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.769+0000 7f553ec00640 1 -- 172.21.15.40:0/2503829969 shutdown_connections 2024-03-08T06:06:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.769+0000 7f553ec00640 1 -- 172.21.15.40:0/2503829969 >> 172.21.15.40:0/2503829969 conn(0x7f553806c740 msgr2=0x7f553806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.769+0000 7f553ec00640 1 -- 172.21.15.40:0/2503829969 shutdown_connections 2024-03-08T06:06:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.769+0000 7f553ec00640 1 -- 172.21.15.40:0/2503829969 wait complete. 2024-03-08T06:06:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.771+0000 7f553ec00640 1 Processor -- start 2024-03-08T06:06:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.771+0000 7f553ec00640 1 -- start start 2024-03-08T06:06:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553ec00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 0x7f553814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553ec00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55380cc160 0x7f55381f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553ec00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55380aa4f0 0x7f55380aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553ec00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380b5e00 con 0x7f5520003f80 2024-03-08T06:06:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553ec00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380b5c80 con 0x7f55380cc160 2024-03-08T06:06:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553ec00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55380b2de0 con 0x7f55380aa4f0 2024-03-08T06:06:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553d800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 0x7f553814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553d800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 0x7f553814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36196/0 (socket says 172.21.15.40:36196) 2024-03-08T06:06:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553d800640 1 -- 172.21.15.40:0/2488048368 learned_addr learned my addr 172.21.15.40:0/2488048368 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553e200640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55380aa4f0 0x7f55380aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553ce00640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55380cc160 0x7f55381f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553d800640 1 -- 172.21.15.40:0/2488048368 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55380aa4f0 msgr2=0x7f55380aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553d800640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55380aa4f0 0x7f55380aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553d800640 1 -- 172.21.15.40:0/2488048368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55380cc160 msgr2=0x7f55381f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553d800640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55380cc160 0x7f55381f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.772+0000 7f553d800640 1 -- 172.21.15.40:0/2488048368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5538147cc0 con 0x7f5520003f80 2024-03-08T06:06:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.773+0000 7f553d800640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 0x7f553814afd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f552c011600 tx=0x7f552c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.773+0000 7f5535600640 1 -- 172.21.15.40:0/2488048368 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c00f920 con 0x7f5520003f80 2024-03-08T06:06:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.773+0000 7f5535600640 1 -- 172.21.15.40:0/2488048368 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552c00fac0 con 0x7f5520003f80 2024-03-08T06:06:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.773+0000 7f5535600640 1 -- 172.21.15.40:0/2488048368 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c00fe70 con 0x7f5520003f80 2024-03-08T06:06:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.773+0000 7f553ec00640 1 -- 172.21.15.40:0/2488048368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5538147fa0 con 0x7f5520003f80 2024-03-08T06:06:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.773+0000 7f553ec00640 1 -- 172.21.15.40:0/2488048368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5538148560 con 0x7f5520003f80 2024-03-08T06:06:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.774+0000 7f553ec00640 1 -- 172.21.15.40:0/2488048368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5538075100 con 0x7f5520003f80 2024-03-08T06:06:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.775+0000 7f5535600640 1 -- 172.21.15.40:0/2488048368 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f552c015460 con 0x7f5520003f80 2024-03-08T06:06:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.776+0000 7f5535600640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f551806b500 0x7f551806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.776+0000 7f5535600640 1 -- 172.21.15.40:0/2488048368 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f552c08a830 con 0x7f5520003f80 2024-03-08T06:06:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.777+0000 7f553ce00640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f551806b500 0x7f551806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.777+0000 7f553ce00640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f551806b500 0x7f551806d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5524034000 tx=0x7f5524007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.781+0000 7f5535600640 1 -- 172.21.15.40:0/2488048368 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f552c019030 con 0x7f5520003f80 2024-03-08T06:06:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.975+0000 7f553ec00640 1 -- 172.21.15.40:0/2488048368 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55380b6250 con 0x7f551806b500 2024-03-08T06:06:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.977+0000 7f5535600640 1 -- 172.21.15.40:0/2488048368 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13521 (secure 0 0 0) 0x7f55380b6250 con 0x7f551806b500 2024-03-08T06:06:30.977 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-03-08T06:06:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.979+0000 7f553ec00640 1 -- 172.21.15.40:0/2488048368 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f551806b500 msgr2=0x7f551806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.979+0000 7f553ec00640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f551806b500 0x7f551806d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5524034000 tx=0x7f5524007000 comp rx=0 tx=0).stop 2024-03-08T06:06:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.979+0000 7f553ec00640 1 -- 172.21.15.40:0/2488048368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 msgr2=0x7f553814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.979+0000 7f553ec00640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 0x7f553814afd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f552c011600 tx=0x7f552c011ac0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.980+0000 7f553ec00640 1 -- 172.21.15.40:0/2488048368 shutdown_connections 2024-03-08T06:06:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.980+0000 7f553ec00640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f551806b500 0x7f551806d9c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.980+0000 7f553ec00640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55380aa4f0 0x7f55380aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.980+0000 7f553ec00640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55380cc160 0x7f55381f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.980+0000 7f553ec00640 1 --2- 172.21.15.40:0/2488048368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5520003f80 0x7f553814afd0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.980+0000 7f553ec00640 1 -- 172.21.15.40:0/2488048368 >> 172.21.15.40:0/2488048368 conn(0x7f553806c740 msgr2=0x7f553815c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.980+0000 7f553ec00640 1 -- 172.21.15.40:0/2488048368 shutdown_connections 2024-03-08T06:06:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:30.980+0000 7f553ec00640 1 -- 172.21.15.40:0/2488048368 wait complete. 2024-03-08T06:06:31.000 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-03-08T06:06:29.536443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279280,"kb_used_data":25684,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754695440,"statfs":{"total":773094113280,"available":772808130560,"internally_reserved":0,"allocated":26300416,"data_stored":16134347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6850374,"num_objects":-1023,"num_object_clones":0,"num_object_copies":-2046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2084,"num_write_kb":-8678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.382625"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":196,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-08T06:06:23.761334+0000","last_change":"2024-03-08T06:03:57.473597+0000","last_active":"2024-03-08T06:06:23.761334+0000","last_peered":"2024-03-08T06:06:23.761334+0000","last_clean":"2024-03-08T06:06:23.761334+0000","last_became_active":"2024-03-08T06:03:57.468593+0000","last_became_peered":"2024-03-08T06:03:57.468593+0000","last_unstale":"2024-03-08T06:06:23.761334+0000","last_undegraded":"2024-03-08T06:06:23.761334+0000","last_fullsized":"2024-03-08T06:06:23.761334+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_clean_scrub_stamp":"2024-03-08T06:03:56.214441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:45:28.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2035712,"data_stored":748556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1077248,"data_stored":152382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":158764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35344,"kb_used_data":4164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336496,"statfs":{"total":96636764160,"available":96600571904,"internally_reserved":0,"allocated":4263936,"data_stored":2810042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43748,"kb_used_data":8408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328092,"statfs":{"total":96636764160,"available":96591966208,"internally_reserved":0,"allocated":8609792,"data_stored":7005029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":158038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41100,"kb_used_data":5760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330740,"statfs":{"total":96636764160,"available":96594677760,"internally_reserved":0,"allocated":5898240,"data_stored":4353282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1724416,"data_stored":748254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:06:31.001 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-08T06:06:31.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:06:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.223+0000 7f2eed600640 1 Processor -- start 2024-03-08T06:06:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.223+0000 7f2eed600640 1 -- start start 2024-03-08T06:06:31.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.224+0000 7f2eed600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee8071f50 0x7f2ee806e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.224+0000 7f2eed600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2ee80b46c0 0x7f2ee815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.224+0000 7f2eed600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee80b4bf0 con 0x7f2ee80cf040 2024-03-08T06:06:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.224+0000 7f2eed600640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee806f400 con 0x7f2ee80b46c0 2024-03-08T06:06:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.224+0000 7f2eed600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee806d730 con 0x7f2ee8071f50 2024-03-08T06:06:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.225+0000 7f2ee6a00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2ee80cf040 0x7f2ee8071840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36438/0 (socket says 172.21.15.40:36438) 2024-03-08T06:06:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.225+0000 7f2ee6a00640 1 -- 172.21.15.40:0/2406519278 learned_addr learned my addr 172.21.15.40:0/2406519278 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.225+0000 7f2edfe00640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee8071f50 0x7f2ee806e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.225+0000 7f2ee7400640 1 --2- 172.21.15.40:0/2406519278 >> v2:172.21.15.53:3300/0 conn(0x7f2ee80b46c0 0x7f2ee815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.225+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723128846 0 0) 0x7f2ee80b4bf0 con 0x7f2ee80cf040 2024-03-08T06:06:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.225+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ecc003520 con 0x7f2ee80cf040 2024-03-08T06:06:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421171433 0 0) 0x7f2ecc003520 con 0x7f2ee80cf040 2024-03-08T06:06:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee8071f50 msgr2=0x7f2ee806e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee8071f50 0x7f2ee806e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 >> v2:172.21.15.53:3300/0 conn(0x7f2ee80b46c0 msgr2=0x7f2ee815ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 --2- 172.21.15.40:0/2406519278 >> v2:172.21.15.53:3300/0 conn(0x7f2ee80b46c0 0x7f2ee815ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee806da10 con 0x7f2ee80cf040 2024-03-08T06:06:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f2ed8001670 con 0x7f2ee80cf040 2024-03-08T06:06:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 >> v1:172.21.15.40:6789/0 conn(0x7f2ee80cf040 legacy=0x7f2ee8071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 0x7f2ecc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc004800 0x7f2ecc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee8071f50 0x7f2ecc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee806d730 con 0x7f2ecc0039b0 2024-03-08T06:06:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc00b9c0 con 0x7f2ecc004800 2024-03-08T06:06:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc00bb90 con 0x7f2ee8071f50 2024-03-08T06:06:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6a00640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 0x7f2ecc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee7400640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee8071f50 0x7f2ecc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2edfe00640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc004800 0x7f2ecc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6a00640 1 -- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee8071f50 msgr2=0x7f2ecc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.226+0000 7f2ee6a00640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee8071f50 0x7f2ecc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.227+0000 7f2ee6a00640 1 -- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc004800 msgr2=0x7f2ecc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.227+0000 7f2ee6a00640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc004800 0x7f2ecc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.227+0000 7f2ee6a00640 1 -- 172.21.15.40:0/2406519278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc003520 con 0x7f2ecc0039b0 2024-03-08T06:06:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.227+0000 7f2edfe00640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ecc004800 0x7f2ecc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:31.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.228+0000 7f2ee6a00640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 0x7f2ecc003dd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c1c0 tx=0x7f2ecc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4bd5d7085983d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:31.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.228+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8007c80 con 0x7f2ecc0039b0 2024-03-08T06:06:31.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.228+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed8030070 con 0x7f2ecc0039b0 2024-03-08T06:06:31.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.228+0000 7f2ee6000640 1 -- 172.21.15.40:0/2406519278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8005d30 con 0x7f2ecc0039b0 2024-03-08T06:06:31.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.228+0000 7f2eed600640 1 -- 172.21.15.40:0/2406519278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 msgr2=0x7f2ecc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:31.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.229+0000 7f2eed600640 1 --2- 172.21.15.40:0/2406519278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 0x7f2ecc003dd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c1c0 tx=0x7f2ecc005830 comp rx=0 tx=0).stop 2024-03-08T06:06:31.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.229+0000 7f2ee7400640 1 -- 172.21.15.40:0/2406519278 reap_dead start 2024-03-08T06:06:31.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.229+0000 7f2eed600640 1 -- 172.21.15.40:0/2406519278 shutdown_connections 2024-03-08T06:06:31.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.229+0000 7f2eed600640 1 -- 172.21.15.40:0/2406519278 >> 172.21.15.40:0/2406519278 conn(0x7f2ee806ae30 msgr2=0x7f2ee806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:31.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.229+0000 7f2eed600640 1 -- 172.21.15.40:0/2406519278 shutdown_connections 2024-03-08T06:06:31.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.229+0000 7f2eed600640 1 -- 172.21.15.40:0/2406519278 wait complete. 2024-03-08T06:06:31.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.231+0000 7f2eed600640 1 Processor -- start 2024-03-08T06:06:31.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.231+0000 7f2eed600640 1 -- start start 2024-03-08T06:06:31.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2eed600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 0x7f2ee8145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2eed600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ee81461e0 0x7f2ee80b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2eed600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee80b0b70 0x7f2ee80b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2eed600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee806e390 con 0x7f2ecc0039b0 2024-03-08T06:06:31.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2eed600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee806e210 con 0x7f2ee81461e0 2024-03-08T06:06:31.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2eed600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee806e510 con 0x7f2ee80b0b70 2024-03-08T06:06:31.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2ee6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 0x7f2ee8145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2ee6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 0x7f2ee8145ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36210/0 (socket says 172.21.15.40:36210) 2024-03-08T06:06:31.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2ee6a00640 1 -- 172.21.15.40:0/4054699975 learned_addr learned my addr 172.21.15.40:0/4054699975 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:31.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2ee6a00640 1 -- 172.21.15.40:0/4054699975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee80b0b70 msgr2=0x7f2ee80b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:31.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2ee6a00640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee80b0b70 0x7f2ee80b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2ee6a00640 1 -- 172.21.15.40:0/4054699975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ee81461e0 msgr2=0x7f2ee80b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:31.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2ee6a00640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ee81461e0 0x7f2ee80b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.232+0000 7f2ee6a00640 1 -- 172.21.15.40:0/4054699975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee81418b0 con 0x7f2ecc0039b0 2024-03-08T06:06:31.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.233+0000 7f2ee6a00640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 0x7f2ee8145ca0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c1c0 tx=0x7f2ed8001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:31.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.233+0000 7f2edea00640 1 -- 172.21.15.40:0/4054699975 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8001bc0 con 0x7f2ecc0039b0 2024-03-08T06:06:31.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.233+0000 7f2edea00640 1 -- 172.21.15.40:0/4054699975 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed8004bf0 con 0x7f2ecc0039b0 2024-03-08T06:06:31.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.233+0000 7f2edea00640 1 -- 172.21.15.40:0/4054699975 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed802de60 con 0x7f2ecc0039b0 2024-03-08T06:06:31.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.233+0000 7f2eed600640 1 -- 172.21.15.40:0/4054699975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee812ebb0 con 0x7f2ecc0039b0 2024-03-08T06:06:31.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.233+0000 7f2eed600640 1 -- 172.21.15.40:0/4054699975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee8141e30 con 0x7f2ecc0039b0 2024-03-08T06:06:31.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.234+0000 7f2eed600640 1 -- 172.21.15.40:0/4054699975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee806e910 con 0x7f2ecc0039b0 2024-03-08T06:06:31.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.235+0000 7f2edea00640 1 -- 172.21.15.40:0/4054699975 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ed8007070 con 0x7f2ecc0039b0 2024-03-08T06:06:31.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.236+0000 7f2edea00640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2ec406b4d0 0x7f2ec406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.236+0000 7f2edea00640 1 -- 172.21.15.40:0/4054699975 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2ed80ab510 con 0x7f2ecc0039b0 2024-03-08T06:06:31.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.236+0000 7f2edfe00640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2ec406b4d0 0x7f2ec406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.237+0000 7f2edfe00640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2ec406b4d0 0x7f2ec406d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2ee81f7060 tx=0x7f2ed0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:31.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.241+0000 7f2edea00640 1 -- 172.21.15.40:0/4054699975 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ee806e910 con 0x7f2ecc0039b0 2024-03-08T06:06:31.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.435+0000 7f2eed600640 1 -- 172.21.15.40:0/4054699975 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ee8146620 con 0x7f2ec406b4d0 2024-03-08T06:06:31.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.437+0000 7f2edea00640 1 -- 172.21.15.40:0/4054699975 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13521 (secure 0 0 0) 0x7f2ee8146620 con 0x7f2ec406b4d0 2024-03-08T06:06:31.437 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:31.437 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-03-08T06:06:31.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 -- 172.21.15.40:0/4054699975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2ec406b4d0 msgr2=0x7f2ec406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:31.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2ec406b4d0 0x7f2ec406d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2ee81f7060 tx=0x7f2ed0008040 comp rx=0 tx=0).stop 2024-03-08T06:06:31.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 -- 172.21.15.40:0/4054699975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 msgr2=0x7f2ee8145ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 0x7f2ee8145ca0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c1c0 tx=0x7f2ed8001970 comp rx=0 tx=0).stop 2024-03-08T06:06:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 -- 172.21.15.40:0/4054699975 shutdown_connections 2024-03-08T06:06:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f2ec406b4d0 0x7f2ec406d990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2ee80b0b70 0x7f2ee80b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ee81461e0 0x7f2ee80b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 --2- 172.21.15.40:0/4054699975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ecc0039b0 0x7f2ee8145ca0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 -- 172.21.15.40:0/4054699975 >> 172.21.15.40:0/4054699975 conn(0x7f2ee806ae30 msgr2=0x7f2ee8071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:31.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 -- 172.21.15.40:0/4054699975 shutdown_connections 2024-03-08T06:06:31.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.439+0000 7f2eed600640 1 -- 172.21.15.40:0/4054699975 wait complete. 2024-03-08T06:06:31.459 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-03-08T06:06:29.536443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279280,"kb_used_data":25684,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754695440,"statfs":{"total":773094113280,"available":772808130560,"internally_reserved":0,"allocated":26300416,"data_stored":16134347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6850374,"num_objects":-1023,"num_object_clones":0,"num_object_copies":-2046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2084,"num_write_kb":-8678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.382625"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":196,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-08T06:06:23.761334+0000","last_change":"2024-03-08T06:03:57.473597+0000","last_active":"2024-03-08T06:06:23.761334+0000","last_peered":"2024-03-08T06:06:23.761334+0000","last_clean":"2024-03-08T06:06:23.761334+0000","last_became_active":"2024-03-08T06:03:57.468593+0000","last_became_peered":"2024-03-08T06:03:57.468593+0000","last_unstale":"2024-03-08T06:06:23.761334+0000","last_undegraded":"2024-03-08T06:06:23.761334+0000","last_fullsized":"2024-03-08T06:06:23.761334+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_clean_scrub_stamp":"2024-03-08T06:03:56.214441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:45:28.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2035712,"data_stored":748556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1077248,"data_stored":152382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":158764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35344,"kb_used_data":4164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336496,"statfs":{"total":96636764160,"available":96600571904,"internally_reserved":0,"allocated":4263936,"data_stored":2810042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43748,"kb_used_data":8408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328092,"statfs":{"total":96636764160,"available":96591966208,"internally_reserved":0,"allocated":8609792,"data_stored":7005029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":158038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41100,"kb_used_data":5760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330740,"statfs":{"total":96636764160,"available":96594677760,"internally_reserved":0,"allocated":5898240,"data_stored":4353282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1724416,"data_stored":748254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:06:31.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T06:06:31.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.684+0000 7fcde1a00640 1 Processor -- start 2024-03-08T06:06:31.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.685+0000 7fcde1a00640 1 -- start start 2024-03-08T06:06:31.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.686+0000 7fcde1a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcddc144030 0x7fcddc0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.686+0000 7fcde1a00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcddc15e580 0x7fcddc06e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.686+0000 7fcde1a00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc15eaf0 con 0x7fcddc06ec50 2024-03-08T06:06:31.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.687+0000 7fcde1a00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc0b3350 con 0x7fcddc15e580 2024-03-08T06:06:31.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.687+0000 7fcde1a00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc15edd0 con 0x7fcddc144030 2024-03-08T06:06:31.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.687+0000 7fcdda000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcddc15e580 0x7fcddc06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.687+0000 7fcdda000640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcddc15e580 0x7fcddc06e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36222/0 (socket says 172.21.15.40:36222) 2024-03-08T06:06:31.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.687+0000 7fcdda000640 1 -- 172.21.15.40:0/1607552513 learned_addr learned my addr 172.21.15.40:0/1607552513 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:31.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.687+0000 7fcddaa00640 1 --2- 172.21.15.40:0/1607552513 >> v2:172.21.15.53:3300/0 conn(0x7fcddc144030 0x7fcddc0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.687+0000 7fcdda000640 1 -- 172.21.15.40:0/1607552513 >> v2:172.21.15.53:3300/0 conn(0x7fcddc144030 msgr2=0x7fcddc0cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:31.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.687+0000 7fcdda000640 1 --2- 172.21.15.40:0/1607552513 >> v2:172.21.15.53:3300/0 conn(0x7fcddc144030 0x7fcddc0cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.687+0000 7fcdda000640 1 -- 172.21.15.40:0/1607552513 >> v1:172.21.15.53:6789/0 conn(0x7fcddc06ec50 legacy=0x7fcddc071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:31.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.687+0000 7fcdda000640 1 -- 172.21.15.40:0/1607552513 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc15f0b0 con 0x7fcddc15e580 2024-03-08T06:06:31.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.687+0000 7fcddaa00640 1 --2- 172.21.15.40:0/1607552513 >> v2:172.21.15.53:3300/0 conn(0x7fcddc144030 0x7fcddc0cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:06:31.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.689+0000 7fcdda000640 1 --2- 172.21.15.40:0/1607552513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc15e580 0x7fcddc06e710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcddc06bc50 tx=0x7fcdc802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d60d09628e69fb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:31.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.689+0000 7fcdd9600640 1 -- 172.21.15.40:0/1607552513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8035020 con 0x7fcddc15e580 2024-03-08T06:06:31.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.689+0000 7fcdd9600640 1 -- 172.21.15.40:0/1607552513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc8031870 con 0x7fcddc15e580 2024-03-08T06:06:31.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.689+0000 7fcdd9600640 1 -- 172.21.15.40:0/1607552513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8031c40 con 0x7fcddc15e580 2024-03-08T06:06:31.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.690+0000 7fcde1a00640 1 -- 172.21.15.40:0/1607552513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc15e580 msgr2=0x7fcddc06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:31.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.690+0000 7fcde1a00640 1 --2- 172.21.15.40:0/1607552513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc15e580 0x7fcddc06e710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcddc06bc50 tx=0x7fcdc802b910 comp rx=0 tx=0).stop 2024-03-08T06:06:31.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.690+0000 7fcde1a00640 1 -- 172.21.15.40:0/1607552513 shutdown_connections 2024-03-08T06:06:31.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.690+0000 7fcde1a00640 1 --2- 172.21.15.40:0/1607552513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc15e580 0x7fcddc06e710 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.690+0000 7fcde1a00640 1 --2- 172.21.15.40:0/1607552513 >> v2:172.21.15.53:3300/0 conn(0x7fcddc144030 0x7fcddc0cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.690+0000 7fcde1a00640 1 -- 172.21.15.40:0/1607552513 >> 172.21.15.40:0/1607552513 conn(0x7fcddc06ae30 msgr2=0x7fcddc06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:31.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.690+0000 7fcde1a00640 1 -- 172.21.15.40:0/1607552513 shutdown_connections 2024-03-08T06:06:31.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.690+0000 7fcde1a00640 1 -- 172.21.15.40:0/1607552513 wait complete. 2024-03-08T06:06:31.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.691+0000 7fcde1a00640 1 Processor -- start 2024-03-08T06:06:31.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.692+0000 7fcde1a00640 1 -- start start 2024-03-08T06:06:31.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcde1a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc06ec50 0x7fcddc1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcde1a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcddc144030 0x7fcddc15d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcde1a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcddc15e580 0x7fcddc15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcde1a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0cf740 con 0x7fcddc06ec50 2024-03-08T06:06:31.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcde1a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0cf5c0 con 0x7fcddc144030 2024-03-08T06:06:31.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcde1a00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc144410 con 0x7fcddc15e580 2024-03-08T06:06:31.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcddaa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc06ec50 0x7fcddc1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcddaa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc06ec50 0x7fcddc1405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36238/0 (socket says 172.21.15.40:36238) 2024-03-08T06:06:31.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcddaa00640 1 -- 172.21.15.40:0/470603426 learned_addr learned my addr 172.21.15.40:0/470603426 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:31.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcddaa00640 1 -- 172.21.15.40:0/470603426 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcddc15e580 msgr2=0x7fcddc15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:31.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcdda000640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcddc144030 0x7fcddc15d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcddb400640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcddc15e580 0x7fcddc15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcddaa00640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcddc15e580 0x7fcddc15a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcddaa00640 1 -- 172.21.15.40:0/470603426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcddc144030 msgr2=0x7fcddc15d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:31.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcddaa00640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcddc144030 0x7fcddc15d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.693+0000 7fcddaa00640 1 -- 172.21.15.40:0/470603426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc15d7d0 con 0x7fcddc06ec50 2024-03-08T06:06:31.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.694+0000 7fcddb400640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcddc15e580 0x7fcddc15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:31.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.694+0000 7fcddaa00640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc06ec50 0x7fcddc1405f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e730 tx=0x7fcdcc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.694+0000 7fcdd2a00640 1 -- 172.21.15.40:0/470603426 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00f040 con 0x7fcddc06ec50 2024-03-08T06:06:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.694+0000 7fcdd2a00640 1 -- 172.21.15.40:0/470603426 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc004230 con 0x7fcddc06ec50 2024-03-08T06:06:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.694+0000 7fcde1a00640 1 -- 172.21.15.40:0/470603426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc140c70 con 0x7fcddc06ec50 2024-03-08T06:06:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.694+0000 7fcdd2a00640 1 -- 172.21.15.40:0/470603426 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00ee40 con 0x7fcddc06ec50 2024-03-08T06:06:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.694+0000 7fcde1a00640 1 -- 172.21.15.40:0/470603426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc1414b0 con 0x7fcddc06ec50 2024-03-08T06:06:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.695+0000 7fcde1a00640 1 -- 172.21.15.40:0/470603426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc067cb0 con 0x7fcddc06ec50 2024-03-08T06:06:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.696+0000 7fcdd2a00640 1 -- 172.21.15.40:0/470603426 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdcc00c6d0 con 0x7fcddc06ec50 2024-03-08T06:06:31.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.697+0000 7fcdd2a00640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcdbc06b4d0 0x7fcdbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:31.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.697+0000 7fcdd2a00640 1 -- 172.21.15.40:0/470603426 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcdcc089cc0 con 0x7fcddc06ec50 2024-03-08T06:06:31.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.698+0000 7fcdda000640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcdbc06b4d0 0x7fcdbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:31.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.698+0000 7fcdda000640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcdbc06b4d0 0x7fcdbc06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8036000 tx=0x7fcdc8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:31.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.702+0000 7fcdd2a00640 1 -- 172.21.15.40:0/470603426 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdcc090080 con 0x7fcddc06ec50 2024-03-08T06:06:31.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.903+0000 7fcde1a00640 1 -- 172.21.15.40:0/470603426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcddc0aeec0 con 0x7fcddc06ec50 2024-03-08T06:06:31.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.905+0000 7fcdd2a00640 1 -- 172.21.15.40:0/470603426 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+12936 (secure 0 0 0) 0x7fcddc140c70 con 0x7fcddc06ec50 2024-03-08T06:06:31.905 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:31.906 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":37,"fsid":"c4e20909-a427-4e44-a3fb-93b068b9d1f6","created":"2024-03-08T06:03:42.311009+0000","modified":"2024-03-08T06:06:26.298666+0000","last_up_change":"2024-03-08T06:03:54.205597+0000","last_in_change":"2024-03-08T06:03:45.860054+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T06:03:55.558230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ea92b754-fbb9-461c-8332-5343a83796c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6801","nonce":671313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6803","nonce":671313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6807","nonce":671313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":671313654},{"type":"v1","addr":"172.21.15.40:6805","nonce":671313654}]},"public_addr":"172.21.15.40:6801/671313654","cluster_addr":"172.21.15.40:6803/671313654","heartbeat_back_addr":"172.21.15.40:6807/671313654","heartbeat_front_addr":"172.21.15.40:6805/671313654","state":["exists","up"]},{"osd":1,"uuid":"022bb897-a67f-4555-a1be-0d81a4e306d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6809","nonce":981988607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6811","nonce":981988607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6816","nonce":981988607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":981988607},{"type":"v1","addr":"172.21.15.40:6813","nonce":981988607}]},"public_addr":"172.21.15.40:6809/981988607","cluster_addr":"172.21.15.40:6811/981988607","heartbeat_back_addr":"172.21.15.40:6816/981988607","heartbeat_front_addr":"172.21.15.40:6813/981988607","state":["exists","up"]},{"osd":2,"uuid":"9296f075-c6f9-416e-99dd-13e3db81e1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6822","nonce":1467838663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6826","nonce":1467838663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6831","nonce":1467838663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1467838663},{"type":"v1","addr":"172.21.15.40:6829","nonce":1467838663}]},"public_addr":"172.21.15.40:6822/1467838663","cluster_addr":"172.21.15.40:6826/1467838663","heartbeat_back_addr":"172.21.15.40:6831/1467838663","heartbeat_front_addr":"172.21.15.40:6829/1467838663","state":["exists","up"]},{"osd":3,"uuid":"2d13fea2-46c6-4f71-9031-cdfe8b87de91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6817","nonce":405718921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6820","nonce":405718921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6827","nonce":405718921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":405718921},{"type":"v1","addr":"172.21.15.40:6823","nonce":405718921}]},"public_addr":"172.21.15.40:6817/405718921","cluster_addr":"172.21.15.40:6820/405718921","heartbeat_back_addr":"172.21.15.40:6827/405718921","heartbeat_front_addr":"172.21.15.40:6823/405718921","state":["exists","up"]},{"osd":4,"uuid":"75452991-232c-4cf9-b699-5c9471173669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6817","nonce":2858249123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6819","nonce":2858249123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6823","nonce":2858249123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2858249123},{"type":"v1","addr":"172.21.15.53:6821","nonce":2858249123}]},"public_addr":"172.21.15.53:6817/2858249123","cluster_addr":"172.21.15.53:6819/2858249123","heartbeat_back_addr":"172.21.15.53:6823/2858249123","heartbeat_front_addr":"172.21.15.53:6821/2858249123","state":["exists","up"]},{"osd":5,"uuid":"c1fdd926-f679-471c-b49a-98bbe23ab014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6801","nonce":3704060731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6803","nonce":3704060731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6807","nonce":3704060731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3704060731},{"type":"v1","addr":"172.21.15.53:6805","nonce":3704060731}]},"public_addr":"172.21.15.53:6801/3704060731","cluster_addr":"172.21.15.53:6803/3704060731","heartbeat_back_addr":"172.21.15.53:6807/3704060731","heartbeat_front_addr":"172.21.15.53:6805/3704060731","state":["exists","up"]},{"osd":6,"uuid":"0104e9cc-be81-45e7-ac1f-e0337a67a53c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6825","nonce":2620899738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6827","nonce":2620899738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6831","nonce":2620899738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2620899738},{"type":"v1","addr":"172.21.15.53:6829","nonce":2620899738}]},"public_addr":"172.21.15.53:6825/2620899738","cluster_addr":"172.21.15.53:6827/2620899738","heartbeat_back_addr":"172.21.15.53:6831/2620899738","heartbeat_front_addr":"172.21.15.53:6829/2620899738","state":["exists","up"]},{"osd":7,"uuid":"884629a7-a723-45c4-890c-b14a34610d7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6809","nonce":597382546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6811","nonce":597382546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6815","nonce":597382546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":597382546},{"type":"v1","addr":"172.21.15.53:6813","nonce":597382546}]},"public_addr":"172.21.15.53:6809/597382546","cluster_addr":"172.21.15.53:6811/597382546","heartbeat_back_addr":"172.21.15.53:6815/597382546","heartbeat_front_addr":"172.21.15.53:6813/597382546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.447300+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.474336+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.173177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.195094+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.256702+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.441446+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:53.265171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T06:03:51.202103+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6836/1742457959":"2024-03-09T06:06:23.147053+0000","172.21.15.40:6834/1742457959":"2024-03-09T06:06:23.147053+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T06:06:31.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 -- 172.21.15.40:0/470603426 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcdbc06b4d0 msgr2=0x7fcdbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:31.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcdbc06b4d0 0x7fcdbc06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8036000 tx=0x7fcdc8008030 comp rx=0 tx=0).stop 2024-03-08T06:06:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 -- 172.21.15.40:0/470603426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc06ec50 msgr2=0x7fcddc1405f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc06ec50 0x7fcddc1405f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e730 tx=0x7fcdcc00ebf0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 -- 172.21.15.40:0/470603426 shutdown_connections 2024-03-08T06:06:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fcdbc06b4d0 0x7fcdbc06d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcddc15e580 0x7fcddc15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcddc144030 0x7fcddc15d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 --2- 172.21.15.40:0/470603426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcddc06ec50 0x7fcddc1405f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 -- 172.21.15.40:0/470603426 >> 172.21.15.40:0/470603426 conn(0x7fcddc06ae30 msgr2=0x7fcddc0b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 -- 172.21.15.40:0/470603426 shutdown_connections 2024-03-08T06:06:31.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:31.908+0000 7fcde1a00640 1 -- 172.21.15.40:0/470603426 wait complete. 2024-03-08T06:06:32.929 INFO:tasks.ceph:Scrubbing osd.0 2024-03-08T06:06:32.929 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:06:33.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.168+0000 7fb06d800640 1 Processor -- start 2024-03-08T06:06:33.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.168+0000 7fb06d800640 1 -- start start 2024-03-08T06:06:33.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.170+0000 7fb06d800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb0680d10c0 0x7fb0680ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.170+0000 7fb06d800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0680d0cd0 con 0x7fb0680d3030 2024-03-08T06:06:33.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.170+0000 7fb06d800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0680d2270 con 0x7fb068161460 2024-03-08T06:06:33.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.170+0000 7fb06d800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0680d2550 con 0x7fb0680d10c0 2024-03-08T06:06:33.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.170+0000 7fb06ce00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb0680d3030 0x7fb0680b55e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36448/0 (socket says 172.21.15.40:36448) 2024-03-08T06:06:33.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.170+0000 7fb06ce00640 1 -- 172.21.15.40:0/1445052887 learned_addr learned my addr 172.21.15.40:0/1445052887 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:33.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.170+0000 7fb067400640 1 --2- 172.21.15.40:0/1445052887 >> v2:172.21.15.53:3300/0 conn(0x7fb0680d10c0 0x7fb0680ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516528018 0 0) 0x7fb060001110 con 0x7fb0680d3030 2024-03-08T06:06:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb04c003520 con 0x7fb0680d3030 2024-03-08T06:06:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2971473244 0 0) 0x7fb04c003520 con 0x7fb0680d3030 2024-03-08T06:06:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 >> v2:172.21.15.53:3300/0 conn(0x7fb0680d10c0 msgr2=0x7fb0680ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 --2- 172.21.15.40:0/1445052887 >> v2:172.21.15.53:3300/0 conn(0x7fb0680d10c0 0x7fb0680ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 >> v1:172.21.15.53:6789/0 conn(0x7fb068161460 legacy=0x7fb0680b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0680d2830 con 0x7fb0680d3030 2024-03-08T06:06:33.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fb060001840 con 0x7fb0680d3030 2024-03-08T06:06:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 >> v1:172.21.15.40:6789/0 conn(0x7fb0680d3030 legacy=0x7fb0680b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 --2- 172.21.15.40:0/1445052887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb04c003a70 0x7fb04c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 --2- 172.21.15.40:0/1445052887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 0x7fb04c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 --2- 172.21.15.40:0/1445052887 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb04c005210 0x7fb04c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680d2550 con 0x7fb04c004800 2024-03-08T06:06:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c00c140 con 0x7fb04c003a70 2024-03-08T06:06:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.171+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c00c340 con 0x7fb04c005210 2024-03-08T06:06:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.172+0000 7fb067400640 1 --2- 172.21.15.40:0/1445052887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 0x7fb04c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.172+0000 7fb067400640 1 -- 172.21.15.40:0/1445052887 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb04c005210 msgr2=0x7fb04c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:33.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.172+0000 7fb067400640 1 --2- 172.21.15.40:0/1445052887 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb04c005210 0x7fb04c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.172+0000 7fb067400640 1 -- 172.21.15.40:0/1445052887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb04c003a70 msgr2=0x7fb04c003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:33.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.172+0000 7fb067400640 1 --2- 172.21.15.40:0/1445052887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb04c003a70 0x7fb04c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.172+0000 7fb067400640 1 -- 172.21.15.40:0/1445052887 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c003520 con 0x7fb04c004800 2024-03-08T06:06:33.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.172+0000 7fb06ce00640 1 -- 172.21.15.40:0/1445052887 reap_dead start 2024-03-08T06:06:33.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.173+0000 7fb067400640 1 --2- 172.21.15.40:0/1445052887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 0x7fb04c004c80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb054009ac0 tx=0x7fb04c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee58f6bc6c1d5c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:33.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.173+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054039070 con 0x7fb04c004800 2024-03-08T06:06:33.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.173+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05402c6b0 con 0x7fb04c004800 2024-03-08T06:06:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.174+0000 7fb066a00640 1 -- 172.21.15.40:0/1445052887 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054009690 con 0x7fb04c004800 2024-03-08T06:06:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.174+0000 7fb06d800640 1 -- 172.21.15.40:0/1445052887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 msgr2=0x7fb04c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.174+0000 7fb06d800640 1 --2- 172.21.15.40:0/1445052887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 0x7fb04c004c80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb054009ac0 tx=0x7fb04c005830 comp rx=0 tx=0).stop 2024-03-08T06:06:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.174+0000 7fb06d800640 1 -- 172.21.15.40:0/1445052887 shutdown_connections 2024-03-08T06:06:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.174+0000 7fb06d800640 1 --2- 172.21.15.40:0/1445052887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 0x7fb04c004c80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.174+0000 7fb06d800640 1 -- 172.21.15.40:0/1445052887 >> 172.21.15.40:0/1445052887 conn(0x7fb06806c7b0 msgr2=0x7fb06806cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.174+0000 7fb06d800640 1 -- 172.21.15.40:0/1445052887 shutdown_connections 2024-03-08T06:06:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.174+0000 7fb06d800640 1 -- 172.21.15.40:0/1445052887 wait complete. 2024-03-08T06:06:33.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.176+0000 7fb06d800640 1 Processor -- start 2024-03-08T06:06:33.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.176+0000 7fb06d800640 1 -- start start 2024-03-08T06:06:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.177+0000 7fb06d800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 0x7fb0681f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.177+0000 7fb06d800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0680b5060 0x7fb068156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.177+0000 7fb06d800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0680b7580 0x7fb0680b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.177+0000 7fb06d800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb068161840 con 0x7fb04c004800 2024-03-08T06:06:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.177+0000 7fb06d800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680cf230 con 0x7fb0680b5060 2024-03-08T06:06:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.177+0000 7fb06d800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680ceb30 con 0x7fb0680b7580 2024-03-08T06:06:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.177+0000 7fb067e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 0x7fb0681f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.177+0000 7fb067e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 0x7fb0681f6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36246/0 (socket says 172.21.15.40:36246) 2024-03-08T06:06:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.177+0000 7fb067e00640 1 -- 172.21.15.40:0/4206575577 learned_addr learned my addr 172.21.15.40:0/4206575577 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.178+0000 7fb06ce00640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0680b7580 0x7fb0680b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.178+0000 7fb067400640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0680b5060 0x7fb068156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.178+0000 7fb067e00640 1 -- 172.21.15.40:0/4206575577 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0680b7580 msgr2=0x7fb0680b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.178+0000 7fb067e00640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0680b7580 0x7fb0680b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.178+0000 7fb067e00640 1 -- 172.21.15.40:0/4206575577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0680b5060 msgr2=0x7fb068156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.178+0000 7fb067e00640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0680b5060 0x7fb068156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.178+0000 7fb067e00640 1 -- 172.21.15.40:0/4206575577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb068156c70 con 0x7fb04c004800 2024-03-08T06:06:33.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.178+0000 7fb06ce00640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0680b7580 0x7fb0680b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:33.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.178+0000 7fb067400640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0680b5060 0x7fb068156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:33.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.178+0000 7fb067e00640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 0x7fb0681f6d70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb05800c840 tx=0x7fb05800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:33.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.178+0000 7fb05fe00640 1 -- 172.21.15.40:0/4206575577 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058003b70 con 0x7fb04c004800 2024-03-08T06:06:33.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.179+0000 7fb05fe00640 1 -- 172.21.15.40:0/4206575577 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb058003d30 con 0x7fb04c004800 2024-03-08T06:06:33.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.179+0000 7fb06d800640 1 -- 172.21.15.40:0/4206575577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb068156f50 con 0x7fb04c004800 2024-03-08T06:06:33.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.179+0000 7fb05fe00640 1 -- 172.21.15.40:0/4206575577 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800aa30 con 0x7fb04c004800 2024-03-08T06:06:33.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.179+0000 7fb06d800640 1 -- 172.21.15.40:0/4206575577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0681f72b0 con 0x7fb04c004800 2024-03-08T06:06:33.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.180+0000 7fb06d800640 1 -- 172.21.15.40:0/4206575577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb068157470 con 0x7fb04c004800 2024-03-08T06:06:33.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.181+0000 7fb05fe00640 1 -- 172.21.15.40:0/4206575577 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb05800abd0 con 0x7fb04c004800 2024-03-08T06:06:33.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.182+0000 7fb05fe00640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb04806b500 0x7fb04806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.182+0000 7fb05fe00640 1 -- 172.21.15.40:0/4206575577 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb058089f10 con 0x7fb04c004800 2024-03-08T06:06:33.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.182+0000 7fb067400640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb04806b500 0x7fb04806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.182+0000 7fb05fe00640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] conn(0x7fb0480713c0 0x7fb0480737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.182+0000 7fb05fe00640 1 -- 172.21.15.40:0/4206575577 --> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb048073e90 con 0x7fb0480713c0 2024-03-08T06:06:33.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.182+0000 7fb05fe00640 1 -- 172.21.15.40:0/4206575577 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb05808c530 con 0x7fb04c004800 2024-03-08T06:06:33.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.182+0000 7fb06ce00640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] conn(0x7fb0480713c0 0x7fb0480737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.183+0000 7fb067400640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb04806b500 0x7fb04806d9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb054036000 tx=0x7fb054030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:33.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.183+0000 7fb06ce00640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] conn(0x7fb0480713c0 0x7fb0480737e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:33.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.184+0000 7fb05fe00640 1 -- 172.21.15.40:0/4206575577 <== osd.0 v2:172.21.15.40:6800/671313654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb048073e90 con 0x7fb0480713c0 2024-03-08T06:06:33.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.198+0000 7fb06d800640 1 -- 172.21.15.40:0/4206575577 --> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb068075090 con 0x7fb0480713c0 2024-03-08T06:06:33.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.199+0000 7fb05fe00640 1 -- 172.21.15.40:0/4206575577 <== osd.0 v2:172.21.15.40:6800/671313654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb068075090 con 0x7fb0480713c0 2024-03-08T06:06:33.199 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:06:33.199 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:06:33.199 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:06:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.199+0000 7fb06d800640 1 -- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] conn(0x7fb0480713c0 msgr2=0x7fb0480737e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.199+0000 7fb06d800640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:6800/671313654,v1:172.21.15.40:6801/671313654] conn(0x7fb0480713c0 0x7fb0480737e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.199+0000 7fb06d800640 1 -- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb04806b500 msgr2=0x7fb04806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.199+0000 7fb06d800640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fb04806b500 0x7fb04806d9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb054036000 tx=0x7fb054030000 comp rx=0 tx=0).stop 2024-03-08T06:06:33.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.199+0000 7fb06d800640 1 -- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 msgr2=0x7fb0681f6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.199+0000 7fb06d800640 1 --2- 172.21.15.40:0/4206575577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04c004800 0x7fb0681f6d70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb05800c840 tx=0x7fb05800cd00 comp rx=0 tx=0).stop 2024-03-08T06:06:33.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.200+0000 7fb06ce00640 1 -- 172.21.15.40:0/4206575577 reap_dead start 2024-03-08T06:06:33.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.200+0000 7fb06d800640 1 -- 172.21.15.40:0/4206575577 shutdown_connections 2024-03-08T06:06:33.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.200+0000 7fb06d800640 1 -- 172.21.15.40:0/4206575577 >> 172.21.15.40:0/4206575577 conn(0x7fb06806c7b0 msgr2=0x7fb0680b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:33.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.200+0000 7fb06d800640 1 -- 172.21.15.40:0/4206575577 shutdown_connections 2024-03-08T06:06:33.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.200+0000 7fb06d800640 1 -- 172.21.15.40:0/4206575577 wait complete. 2024-03-08T06:06:33.219 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-08T06:06:33.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.441+0000 7ff2a8e00640 1 Processor -- start 2024-03-08T06:06:33.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.441+0000 7ff2a8e00640 1 -- start start 2024-03-08T06:06:33.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.443+0000 7ff2a8e00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2a40cd730 0x7ff2a40cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.443+0000 7ff2a8e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff2a40717d0 0x7ff2a4071c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.443+0000 7ff2a8e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a40cc690 con 0x7ff2a4145250 2024-03-08T06:06:33.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.443+0000 7ff2a8e00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a4072180 con 0x7ff2a40cd730 2024-03-08T06:06:33.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.443+0000 7ff2a8e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a40cc970 con 0x7ff2a40717d0 2024-03-08T06:06:33.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.443+0000 7ff2a1400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2a40cd730 0x7ff2a40cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.443+0000 7ff2a1400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2a40cd730 0x7ff2a40cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36260/0 (socket says 172.21.15.40:36260) 2024-03-08T06:06:33.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.443+0000 7ff2a1400640 1 -- 172.21.15.40:0/3932173595 learned_addr learned my addr 172.21.15.40:0/3932173595 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:33.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.444+0000 7ff2a2800640 1 --2- 172.21.15.40:0/3932173595 >> v2:172.21.15.53:3300/0 conn(0x7ff2a40717d0 0x7ff2a4071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.444+0000 7ff2a1400640 1 -- 172.21.15.40:0/3932173595 >> v2:172.21.15.53:3300/0 conn(0x7ff2a40717d0 msgr2=0x7ff2a4071c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:33.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.444+0000 7ff2a1400640 1 --2- 172.21.15.40:0/3932173595 >> v2:172.21.15.53:3300/0 conn(0x7ff2a40717d0 0x7ff2a4071c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.444+0000 7ff2a1400640 1 -- 172.21.15.40:0/3932173595 >> v1:172.21.15.53:6789/0 conn(0x7ff2a4145250 legacy=0x7ff2a406eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:33.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.444+0000 7ff2a1400640 1 -- 172.21.15.40:0/3932173595 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40ccc50 con 0x7ff2a40cd730 2024-03-08T06:06:33.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.445+0000 7ff2a1400640 1 --2- 172.21.15.40:0/3932173595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a40cd730 0x7ff2a40cc0b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40cf3c0 tx=0x7ff28c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0bb2fa17fadae server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:33.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.445+0000 7ff2a0a00640 1 -- 172.21.15.40:0/3932173595 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c035020 con 0x7ff2a40cd730 2024-03-08T06:06:33.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.445+0000 7ff2a0a00640 1 -- 172.21.15.40:0/3932173595 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c031870 con 0x7ff2a40cd730 2024-03-08T06:06:33.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.445+0000 7ff2a0a00640 1 -- 172.21.15.40:0/3932173595 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c031c40 con 0x7ff2a40cd730 2024-03-08T06:06:33.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.446+0000 7ff2a8e00640 1 -- 172.21.15.40:0/3932173595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a40cd730 msgr2=0x7ff2a40cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.446+0000 7ff2a8e00640 1 --2- 172.21.15.40:0/3932173595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a40cd730 0x7ff2a40cc0b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40cf3c0 tx=0x7ff28c02b910 comp rx=0 tx=0).stop 2024-03-08T06:06:33.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.446+0000 7ff2a8e00640 1 -- 172.21.15.40:0/3932173595 shutdown_connections 2024-03-08T06:06:33.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.446+0000 7ff2a8e00640 1 --2- 172.21.15.40:0/3932173595 >> v2:172.21.15.53:3300/0 conn(0x7ff2a40717d0 0x7ff2a4071c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.446+0000 7ff2a8e00640 1 --2- 172.21.15.40:0/3932173595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a40cd730 0x7ff2a40cc0b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.446+0000 7ff2a8e00640 1 -- 172.21.15.40:0/3932173595 >> 172.21.15.40:0/3932173595 conn(0x7ff2a406ae30 msgr2=0x7ff2a406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:33.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.446+0000 7ff2a8e00640 1 -- 172.21.15.40:0/3932173595 shutdown_connections 2024-03-08T06:06:33.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.446+0000 7ff2a8e00640 1 -- 172.21.15.40:0/3932173595 wait complete. 2024-03-08T06:06:33.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.448+0000 7ff2a8e00640 1 Processor -- start 2024-03-08T06:06:33.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.448+0000 7ff2a8e00640 1 -- start start 2024-03-08T06:06:33.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.449+0000 7ff2a8e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff2a40717d0 0x7ff2a413c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.449+0000 7ff2a8e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a4145250 0x7ff2a413ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.449+0000 7ff2a8e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a413f340 0x7ff2a4141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.449+0000 7ff2a8e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a4145630 con 0x7ff2a413f340 2024-03-08T06:06:33.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.449+0000 7ff2a8e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40cd210 con 0x7ff2a4145250 2024-03-08T06:06:33.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.449+0000 7ff2a8e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a41457b0 con 0x7ff2a40717d0 2024-03-08T06:06:33.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.449+0000 7ff2a2800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a413f340 0x7ff2a4141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.449+0000 7ff2a2800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a413f340 0x7ff2a4141c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36268/0 (socket says 172.21.15.40:36268) 2024-03-08T06:06:33.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.449+0000 7ff2a2800640 1 -- 172.21.15.40:0/2878460854 learned_addr learned my addr 172.21.15.40:0/2878460854 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:33.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.450+0000 7ff2a1e00640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff2a40717d0 0x7ff2a413c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.450+0000 7ff2a1400640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a4145250 0x7ff2a413ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.450+0000 7ff2a2800640 1 -- 172.21.15.40:0/2878460854 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff2a40717d0 msgr2=0x7ff2a413c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.450+0000 7ff2a2800640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff2a40717d0 0x7ff2a413c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.450+0000 7ff2a2800640 1 -- 172.21.15.40:0/2878460854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a4145250 msgr2=0x7ff2a413ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.450+0000 7ff2a2800640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a4145250 0x7ff2a413ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.450+0000 7ff2a2800640 1 -- 172.21.15.40:0/2878460854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a4142970 con 0x7ff2a413f340 2024-03-08T06:06:33.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.450+0000 7ff2a2800640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a413f340 0x7ff2a4141c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff29c007e70 tx=0x7ff29c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:33.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.450+0000 7ff29a000640 1 -- 172.21.15.40:0/2878460854 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c014620 con 0x7ff2a413f340 2024-03-08T06:06:33.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.450+0000 7ff29a000640 1 -- 172.21.15.40:0/2878460854 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c019070 con 0x7ff2a413f340 2024-03-08T06:06:33.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.451+0000 7ff29a000640 1 -- 172.21.15.40:0/2878460854 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c014a20 con 0x7ff2a413f340 2024-03-08T06:06:33.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.451+0000 7ff2a8e00640 1 -- 172.21.15.40:0/2878460854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a4142c50 con 0x7ff2a413f340 2024-03-08T06:06:33.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.451+0000 7ff2a8e00640 1 -- 172.21.15.40:0/2878460854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a415c2f0 con 0x7ff2a413f340 2024-03-08T06:06:33.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.452+0000 7ff2a8e00640 1 -- 172.21.15.40:0/2878460854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a4067cb0 con 0x7ff2a413f340 2024-03-08T06:06:33.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.453+0000 7ff29a000640 1 -- 172.21.15.40:0/2878460854 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff29c014bc0 con 0x7ff2a413f340 2024-03-08T06:06:33.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.453+0000 7ff29a000640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff28006b4d0 0x7ff28006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.454+0000 7ff29a000640 1 -- 172.21.15.40:0/2878460854 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff29c08a890 con 0x7ff2a413f340 2024-03-08T06:06:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.454+0000 7ff2a1e00640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff28006b4d0 0x7ff28006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.455+0000 7ff2a1e00640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff28006b4d0 0x7ff28006d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff294006fd0 tx=0x7ff294008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:33.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.458+0000 7ff29a000640 1 -- 172.21.15.40:0/2878460854 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff29c059980 con 0x7ff2a413f340 2024-03-08T06:06:33.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.658+0000 7ff2a8e00640 1 -- 172.21.15.40:0/2878460854 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2a415c530 con 0x7ff28006b4d0 2024-03-08T06:06:33.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.659+0000 7ff29a000640 1 -- 172.21.15.40:0/2878460854 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff29400be30 con 0x7ff28006b4d0 2024-03-08T06:06:33.659 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2024-03-08T06:06:33.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.661+0000 7ff2a8e00640 1 -- 172.21.15.40:0/2878460854 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff28006b4d0 msgr2=0x7ff28006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.661+0000 7ff2a8e00640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff28006b4d0 0x7ff28006d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff294006fd0 tx=0x7ff294008040 comp rx=0 tx=0).stop 2024-03-08T06:06:33.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.661+0000 7ff2a8e00640 1 -- 172.21.15.40:0/2878460854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a413f340 msgr2=0x7ff2a4141c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.661+0000 7ff2a8e00640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a413f340 0x7ff2a4141c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff29c007e70 tx=0x7ff29c00e4b0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.661+0000 7ff2a8e00640 1 -- 172.21.15.40:0/2878460854 shutdown_connections 2024-03-08T06:06:33.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.661+0000 7ff2a8e00640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7ff28006b4d0 0x7ff28006d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.661+0000 7ff2a8e00640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a413f340 0x7ff2a4141c40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.661+0000 7ff2a8e00640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2a4145250 0x7ff2a413ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.661+0000 7ff2a8e00640 1 --2- 172.21.15.40:0/2878460854 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff2a40717d0 0x7ff2a413c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.661+0000 7ff2a8e00640 1 -- 172.21.15.40:0/2878460854 >> 172.21.15.40:0/2878460854 conn(0x7ff2a406ae30 msgr2=0x7ff2a40af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:33.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.661+0000 7ff2a8e00640 1 -- 172.21.15.40:0/2878460854 shutdown_connections 2024-03-08T06:06:33.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.662+0000 7ff2a8e00640 1 -- 172.21.15.40:0/2878460854 wait complete. 2024-03-08T06:06:33.681 INFO:tasks.ceph:Scrubbing osd.1 2024-03-08T06:06:33.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:06:33.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.912+0000 7f879d400640 1 Processor -- start 2024-03-08T06:06:33.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.912+0000 7f879d400640 1 -- start start 2024-03-08T06:06:33.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.913+0000 7f879d400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f87980ce8f0 0x7f87980d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.913+0000 7f879d400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87980d3500 0x7f87980b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.913+0000 7f879d400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87980b5740 con 0x7f87980cecd0 2024-03-08T06:06:33.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.913+0000 7f879d400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87980b7220 con 0x7f87980d3500 2024-03-08T06:06:33.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.913+0000 7f879d400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87980b5a20 con 0x7f87980ce8f0 2024-03-08T06:06:33.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.914+0000 7f878f400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87980d3500 0x7f87980b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.914+0000 7f878f400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f87980d3500 0x7f87980b6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36272/0 (socket says 172.21.15.40:36272) 2024-03-08T06:06:33.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.914+0000 7f878f400640 1 -- 172.21.15.40:0/3681854729 learned_addr learned my addr 172.21.15.40:0/3681854729 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:33.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.914+0000 7f8797e00640 1 --2- 172.21.15.40:0/3681854729 >> v2:172.21.15.53:3300/0 conn(0x7f87980ce8f0 0x7f87980d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.914+0000 7f878f400640 1 -- 172.21.15.40:0/3681854729 >> v2:172.21.15.53:3300/0 conn(0x7f87980ce8f0 msgr2=0x7f87980d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:33.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.914+0000 7f878f400640 1 --2- 172.21.15.40:0/3681854729 >> v2:172.21.15.53:3300/0 conn(0x7f87980ce8f0 0x7f87980d2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.914+0000 7f878f400640 1 -- 172.21.15.40:0/3681854729 >> v1:172.21.15.40:6789/0 conn(0x7f87980cecd0 legacy=0x7f8798161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.914+0000 7f878f400640 1 -- 172.21.15.40:0/3681854729 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87980b5d00 con 0x7f87980d3500 2024-03-08T06:06:33.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.915+0000 7f878f400640 1 --2- 172.21.15.40:0/3681854729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87980d3500 0x7f87980b6ce0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f879808c0d0 tx=0x7f878002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408426c2c60a047f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:33.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.915+0000 7f8797400640 1 -- 172.21.15.40:0/3681854729 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8780034070 con 0x7f87980d3500 2024-03-08T06:06:33.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.916+0000 7f8797400640 1 -- 172.21.15.40:0/3681854729 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8780030860 con 0x7f87980d3500 2024-03-08T06:06:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.916+0000 7f8797400640 1 -- 172.21.15.40:0/3681854729 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8780030c30 con 0x7f87980d3500 2024-03-08T06:06:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.916+0000 7f879d400640 1 -- 172.21.15.40:0/3681854729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87980d3500 msgr2=0x7f87980b6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.916+0000 7f879d400640 1 --2- 172.21.15.40:0/3681854729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87980d3500 0x7f87980b6ce0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f879808c0d0 tx=0x7f878002b9a0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.916+0000 7f879d400640 1 -- 172.21.15.40:0/3681854729 shutdown_connections 2024-03-08T06:06:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.916+0000 7f879d400640 1 --2- 172.21.15.40:0/3681854729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87980d3500 0x7f87980b6ce0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.916+0000 7f879d400640 1 --2- 172.21.15.40:0/3681854729 >> v2:172.21.15.53:3300/0 conn(0x7f87980ce8f0 0x7f87980d2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.916+0000 7f879d400640 1 -- 172.21.15.40:0/3681854729 >> 172.21.15.40:0/3681854729 conn(0x7f879806c4c0 msgr2=0x7f879806c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.916+0000 7f879d400640 1 -- 172.21.15.40:0/3681854729 shutdown_connections 2024-03-08T06:06:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.917+0000 7f879d400640 1 -- 172.21.15.40:0/3681854729 wait complete. 2024-03-08T06:06:33.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.918+0000 7f879d400640 1 Processor -- start 2024-03-08T06:06:33.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.918+0000 7f879d400640 1 -- start start 2024-03-08T06:06:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.919+0000 7f879d400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87980ce8f0 0x7f87981f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.919+0000 7f879d400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87980cecd0 0x7f87980c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.919+0000 7f879d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87980d3500 0x7f87980c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.919+0000 7f879d400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879815e250 con 0x7f87980ce8f0 2024-03-08T06:06:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.919+0000 7f879d400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879815e0d0 con 0x7f87980d3500 2024-03-08T06:06:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.919+0000 7f879d400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87980cf160 con 0x7f87980cecd0 2024-03-08T06:06:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.919+0000 7f8797e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87980ce8f0 0x7f87981f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.920+0000 7f8797e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87980ce8f0 0x7f87981f40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36282/0 (socket says 172.21.15.40:36282) 2024-03-08T06:06:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.920+0000 7f8797e00640 1 -- 172.21.15.40:0/3632826352 learned_addr learned my addr 172.21.15.40:0/3632826352 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.920+0000 7f878f400640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87980cecd0 0x7f87980c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.920+0000 7f879ca00640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87980d3500 0x7f87980c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.920+0000 7f8797e00640 1 -- 172.21.15.40:0/3632826352 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87980cecd0 msgr2=0x7f87980c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.920+0000 7f8797e00640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87980cecd0 0x7f87980c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.920+0000 7f8797e00640 1 -- 172.21.15.40:0/3632826352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87980d3500 msgr2=0x7f87980c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.920+0000 7f8797e00640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87980d3500 0x7f87980c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.920+0000 7f8797e00640 1 -- 172.21.15.40:0/3632826352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87980c8e10 con 0x7f87980ce8f0 2024-03-08T06:06:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.920+0000 7f878f400640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87980cecd0 0x7f87980c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.920+0000 7f8797e00640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87980ce8f0 0x7f87981f40f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f878800ccd0 tx=0x7f878800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.921+0000 7f8794c00640 1 -- 172.21.15.40:0/3632826352 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788003990 con 0x7f87980ce8f0 2024-03-08T06:06:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.921+0000 7f8794c00640 1 -- 172.21.15.40:0/3632826352 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788003b30 con 0x7f87980ce8f0 2024-03-08T06:06:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.921+0000 7f8794c00640 1 -- 172.21.15.40:0/3632826352 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788004020 con 0x7f87980ce8f0 2024-03-08T06:06:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.921+0000 7f879d400640 1 -- 172.21.15.40:0/3632826352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87980c90f0 con 0x7f87980ce8f0 2024-03-08T06:06:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.921+0000 7f879d400640 1 -- 172.21.15.40:0/3632826352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8798147c00 con 0x7f87980ce8f0 2024-03-08T06:06:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.922+0000 7f879d400640 1 -- 172.21.15.40:0/3632826352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8798147710 con 0x7f87980ce8f0 2024-03-08T06:06:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.923+0000 7f8794c00640 1 -- 172.21.15.40:0/3632826352 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87880041c0 con 0x7f87980ce8f0 2024-03-08T06:06:33.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.924+0000 7f8794c00640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f877406b4d0 0x7f877406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.924+0000 7f8794c00640 1 -- 172.21.15.40:0/3632826352 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f878808a3d0 con 0x7f87980ce8f0 2024-03-08T06:06:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.924+0000 7f878f400640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f877406b4d0 0x7f877406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.924+0000 7f8794c00640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] conn(0x7f8774071390 0x7f87740737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.924+0000 7f8794c00640 1 -- 172.21.15.40:0/3632826352 --> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f878800de50 con 0x7f8774071390 2024-03-08T06:06:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.924+0000 7f8794c00640 1 -- 172.21.15.40:0/3632826352 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8788091050 con 0x7f87980ce8f0 2024-03-08T06:06:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.924+0000 7f879ca00640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] conn(0x7f8774071390 0x7f87740737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.925+0000 7f878f400640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f877406b4d0 0x7f877406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8780036000 tx=0x7f878003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.925+0000 7f879ca00640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] conn(0x7f8774071390 0x7f87740737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.926+0000 7f8794c00640 1 -- 172.21.15.40:0/3632826352 <== osd.1 v2:172.21.15.40:6808/981988607 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f878800de50 con 0x7f8774071390 2024-03-08T06:06:33.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.941+0000 7f879d400640 1 -- 172.21.15.40:0/3632826352 --> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8798075090 con 0x7f8774071390 2024-03-08T06:06:33.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.941+0000 7f8794c00640 1 -- 172.21.15.40:0/3632826352 <== osd.1 v2:172.21.15.40:6808/981988607 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8798075090 con 0x7f8774071390 2024-03-08T06:06:33.941 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:06:33.941 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:06:33.941 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:06:33.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.941+0000 7f879d400640 1 -- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] conn(0x7f8774071390 msgr2=0x7f87740737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.941+0000 7f879d400640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:6808/981988607,v1:172.21.15.40:6809/981988607] conn(0x7f8774071390 0x7f87740737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.942+0000 7f879d400640 1 -- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f877406b4d0 msgr2=0x7f877406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.942+0000 7f879d400640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f877406b4d0 0x7f877406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8780036000 tx=0x7f878003b040 comp rx=0 tx=0).stop 2024-03-08T06:06:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.942+0000 7f879d400640 1 -- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87980ce8f0 msgr2=0x7f87981f40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.942+0000 7f879d400640 1 --2- 172.21.15.40:0/3632826352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87980ce8f0 0x7f87981f40f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f878800ccd0 tx=0x7f878800a5c0 comp rx=0 tx=0).stop 2024-03-08T06:06:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.942+0000 7f879ca00640 1 -- 172.21.15.40:0/3632826352 reap_dead start 2024-03-08T06:06:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.942+0000 7f879d400640 1 -- 172.21.15.40:0/3632826352 shutdown_connections 2024-03-08T06:06:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.942+0000 7f879d400640 1 -- 172.21.15.40:0/3632826352 >> 172.21.15.40:0/3632826352 conn(0x7f879806c4c0 msgr2=0x7f87980cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.942+0000 7f879d400640 1 -- 172.21.15.40:0/3632826352 shutdown_connections 2024-03-08T06:06:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:33.942+0000 7f879d400640 1 -- 172.21.15.40:0/3632826352 wait complete. 2024-03-08T06:06:33.959 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-08T06:06:34.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.184+0000 7fbf4b000640 1 Processor -- start 2024-03-08T06:06:34.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.185+0000 7fbf4b000640 1 -- start start 2024-03-08T06:06:34.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.186+0000 7fbf4b000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf4409aa80 0x7fbf440cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.186+0000 7fbf4b000640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf440b2470 0x7fbf440d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.186+0000 7fbf4b000640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf440d16b0 con 0x7fbf44099d40 2024-03-08T06:06:34.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.186+0000 7fbf4b000640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4406c8a0 con 0x7fbf440b2470 2024-03-08T06:06:34.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.186+0000 7fbf4b000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4406cb80 con 0x7fbf4409aa80 2024-03-08T06:06:34.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.186+0000 7fbf43e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbf44099d40 0x7fbf4409a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36478/0 (socket says 172.21.15.40:36478) 2024-03-08T06:06:34.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.186+0000 7fbf43e00640 1 -- 172.21.15.40:0/421446699 learned_addr learned my addr 172.21.15.40:0/421446699 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:34.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf43400640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf4409aa80 0x7fbf440cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf49000640 1 --2- 172.21.15.40:0/421446699 >> v2:172.21.15.53:3300/0 conn(0x7fbf440b2470 0x7fbf440d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714675409 0 0) 0x7fbf440d16b0 con 0x7fbf44099d40 2024-03-08T06:06:34.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf28003520 con 0x7fbf44099d40 2024-03-08T06:06:34.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3854105177 0 0) 0x7fbf28003520 con 0x7fbf44099d40 2024-03-08T06:06:34.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf4409aa80 msgr2=0x7fbf440cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:34.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf42a00640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf4409aa80 0x7fbf440cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 >> v2:172.21.15.53:3300/0 conn(0x7fbf440b2470 msgr2=0x7fbf440d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:34.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf42a00640 1 --2- 172.21.15.40:0/421446699 >> v2:172.21.15.53:3300/0 conn(0x7fbf440b2470 0x7fbf440d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4406ce60 con 0x7fbf44099d40 2024-03-08T06:06:34.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7fbf340019f0 con 0x7fbf44099d40 2024-03-08T06:06:34.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.187+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 >> v1:172.21.15.40:6789/0 conn(0x7fbf44099d40 legacy=0x7fbf4409a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf42a00640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf280039b0 0x7fbf28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf42a00640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf4409aa80 0x7fbf28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf42a00640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf28004d40 0x7fbf280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4406c8a0 con 0x7fbf28004d40 2024-03-08T06:06:34.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4406cb80 con 0x7fbf280039b0 2024-03-08T06:06:34.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2800b9c0 con 0x7fbf4409aa80 2024-03-08T06:06:34.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf49000640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf28004d40 0x7fbf280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf43400640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf4409aa80 0x7fbf28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf43e00640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf280039b0 0x7fbf28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf49000640 1 -- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf4409aa80 msgr2=0x7fbf28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:34.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf49000640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf4409aa80 0x7fbf28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf49000640 1 -- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf280039b0 msgr2=0x7fbf28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:34.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf49000640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf280039b0 0x7fbf28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf49000640 1 -- 172.21.15.40:0/421446699 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf28003520 con 0x7fbf28004d40 2024-03-08T06:06:34.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.188+0000 7fbf43e00640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf280039b0 0x7fbf28003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:34.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.189+0000 7fbf49000640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf28004d40 0x7fbf280051c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009ac0 tx=0x7fbf28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9623534d7a04bab server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:34.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.190+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c039070 con 0x7fbf28004d40 2024-03-08T06:06:34.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.190+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3c02c710 con 0x7fbf28004d40 2024-03-08T06:06:34.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.190+0000 7fbf42a00640 1 -- 172.21.15.40:0/421446699 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c009690 con 0x7fbf28004d40 2024-03-08T06:06:34.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.190+0000 7fbf4b000640 1 -- 172.21.15.40:0/421446699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf28004d40 msgr2=0x7fbf280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.190+0000 7fbf4b000640 1 --2- 172.21.15.40:0/421446699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf28004d40 0x7fbf280051c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009ac0 tx=0x7fbf28005830 comp rx=0 tx=0).stop 2024-03-08T06:06:34.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.190+0000 7fbf49000640 1 -- 172.21.15.40:0/421446699 reap_dead start 2024-03-08T06:06:34.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.190+0000 7fbf4b000640 1 -- 172.21.15.40:0/421446699 shutdown_connections 2024-03-08T06:06:34.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.190+0000 7fbf4b000640 1 -- 172.21.15.40:0/421446699 >> 172.21.15.40:0/421446699 conn(0x7fbf440b3480 msgr2=0x7fbf440b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:34.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.191+0000 7fbf4b000640 1 -- 172.21.15.40:0/421446699 shutdown_connections 2024-03-08T06:06:34.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.191+0000 7fbf4b000640 1 -- 172.21.15.40:0/421446699 wait complete. 2024-03-08T06:06:34.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.192+0000 7fbf4b000640 1 Processor -- start 2024-03-08T06:06:34.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.193+0000 7fbf4b000640 1 -- start start 2024-03-08T06:06:34.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf4b000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf28004d40 0x7fbf441f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf4b000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf44160160 0x7fbf441f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf4b000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf440c8a60 0x7fbf440c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf4b000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf440cfab0 con 0x7fbf44160160 2024-03-08T06:06:34.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf4b000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf440b2900 con 0x7fbf440c8a60 2024-03-08T06:06:34.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf4b000640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf440cfc30 con 0x7fbf28004d40 2024-03-08T06:06:34.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf43400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf44160160 0x7fbf441f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf43400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf44160160 0x7fbf441f0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36308/0 (socket says 172.21.15.40:36308) 2024-03-08T06:06:34.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf43400640 1 -- 172.21.15.40:0/589951442 learned_addr learned my addr 172.21.15.40:0/589951442 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:34.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf43400640 1 -- 172.21.15.40:0/589951442 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf28004d40 msgr2=0x7fbf441f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf43e00640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf28004d40 0x7fbf441f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf43400640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf28004d40 0x7fbf441f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf43400640 1 -- 172.21.15.40:0/589951442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf440c8a60 msgr2=0x7fbf440c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf43400640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf440c8a60 0x7fbf440c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.194+0000 7fbf43400640 1 -- 172.21.15.40:0/589951442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf440cb9c0 con 0x7fbf44160160 2024-03-08T06:06:34.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.195+0000 7fbf43400640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf44160160 0x7fbf441f0eb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c011630 tx=0x7fbf2c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:34.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.195+0000 7fbf3be00640 1 -- 172.21.15.40:0/589951442 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c015870 con 0x7fbf44160160 2024-03-08T06:06:34.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.195+0000 7fbf3be00640 1 -- 172.21.15.40:0/589951442 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c015a10 con 0x7fbf44160160 2024-03-08T06:06:34.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.195+0000 7fbf3be00640 1 -- 172.21.15.40:0/589951442 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c015dc0 con 0x7fbf44160160 2024-03-08T06:06:34.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.195+0000 7fbf4b000640 1 -- 172.21.15.40:0/589951442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf440cadb0 con 0x7fbf44160160 2024-03-08T06:06:34.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.195+0000 7fbf4b000640 1 -- 172.21.15.40:0/589951442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf440cb370 con 0x7fbf44160160 2024-03-08T06:06:34.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.197+0000 7fbf4b000640 1 -- 172.21.15.40:0/589951442 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf44067cb0 con 0x7fbf44160160 2024-03-08T06:06:34.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.198+0000 7fbf3be00640 1 -- 172.21.15.40:0/589951442 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf2c00f490 con 0x7fbf44160160 2024-03-08T06:06:34.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.199+0000 7fbf3be00640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbf2006b4d0 0x7fbf2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.199+0000 7fbf3be00640 1 -- 172.21.15.40:0/589951442 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbf2c08a8e0 con 0x7fbf44160160 2024-03-08T06:06:34.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.199+0000 7fbf43e00640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbf2006b4d0 0x7fbf2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.200+0000 7fbf43e00640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbf2006b4d0 0x7fbf2006d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbf34001b60 tx=0x7fbf34009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:34.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.204+0000 7fbf3be00640 1 -- 172.21.15.40:0/589951442 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf2c019030 con 0x7fbf44160160 2024-03-08T06:06:34.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.405+0000 7fbf4b000640 1 -- 172.21.15.40:0/589951442 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf440c6510 con 0x7fbf2006b4d0 2024-03-08T06:06:34.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.406+0000 7fbf3be00640 1 -- 172.21.15.40:0/589951442 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbf440c6510 con 0x7fbf2006b4d0 2024-03-08T06:06:34.406 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2024-03-08T06:06:34.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 -- 172.21.15.40:0/589951442 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbf2006b4d0 msgr2=0x7fbf2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbf2006b4d0 0x7fbf2006d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbf34001b60 tx=0x7fbf34009040 comp rx=0 tx=0).stop 2024-03-08T06:06:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 -- 172.21.15.40:0/589951442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf44160160 msgr2=0x7fbf441f0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf44160160 0x7fbf441f0eb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c011630 tx=0x7fbf2c011af0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 -- 172.21.15.40:0/589951442 shutdown_connections 2024-03-08T06:06:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fbf2006b4d0 0x7fbf2006d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf440c8a60 0x7fbf440c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf44160160 0x7fbf441f0eb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 --2- 172.21.15.40:0/589951442 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf28004d40 0x7fbf441f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 -- 172.21.15.40:0/589951442 >> 172.21.15.40:0/589951442 conn(0x7fbf440b3480 msgr2=0x7fbf4409a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 -- 172.21.15.40:0/589951442 shutdown_connections 2024-03-08T06:06:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.409+0000 7fbf4b000640 1 -- 172.21.15.40:0/589951442 wait complete. 2024-03-08T06:06:34.428 INFO:tasks.ceph:Scrubbing osd.2 2024-03-08T06:06:34.428 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:06:34.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.653+0000 7f8a85200640 1 Processor -- start 2024-03-08T06:06:34.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.653+0000 7f8a85200640 1 -- start start 2024-03-08T06:06:34.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.655+0000 7f8a85200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8a8014b4b0 0x7f8a8006d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.655+0000 7f8a85200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8a8006da60 0x7f8a800b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.655+0000 7f8a85200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8006dfc0 con 0x7f8a800b38a0 2024-03-08T06:06:34.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.655+0000 7f8a85200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8006e580 con 0x7f8a8014b4b0 2024-03-08T06:06:34.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.655+0000 7f8a85200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8006e2a0 con 0x7f8a8006da60 2024-03-08T06:06:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.656+0000 7f8a7f400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8a8014b4b0 0x7f8a8006d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.656+0000 7f8a76a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8a8006da60 0x7f8a800b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.656+0000 7f8a76a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8a8006da60 0x7f8a800b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:54696/0 (socket says 172.21.15.40:54696) 2024-03-08T06:06:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.656+0000 7f8a7f400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8a8014b4b0 0x7f8a8006d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:49716/0 (socket says 172.21.15.40:49716) 2024-03-08T06:06:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.656+0000 7f8a76a00640 1 -- 172.21.15.40:0/1853839694 learned_addr learned my addr 172.21.15.40:0/1853839694 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.656+0000 7f8a7fe00640 1 --1- 172.21.15.40:0/1853839694 >> v1:172.21.15.53:6789/0 conn(0x7f8a800b38a0 0x7f8a80071520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.40:55578/0 (socket says 172.21.15.40:55578) 2024-03-08T06:06:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.657+0000 7f8a76a00640 1 -- 172.21.15.40:0/1853839694 >> v2:172.21.15.53:3300/0 conn(0x7f8a8014b4b0 msgr2=0x7f8a8006d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:34.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.657+0000 7f8a76a00640 1 --2- 172.21.15.40:0/1853839694 >> v2:172.21.15.53:3300/0 conn(0x7f8a8014b4b0 0x7f8a8006d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.657+0000 7f8a76a00640 1 -- 172.21.15.40:0/1853839694 >> v1:172.21.15.53:6789/0 conn(0x7f8a800b38a0 legacy=0x7f8a80071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.657+0000 7f8a76a00640 1 -- 172.21.15.40:0/1853839694 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a800b2b60 con 0x7f8a8006da60 2024-03-08T06:06:34.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.658+0000 7f8a76a00640 1 --2- 172.21.15.40:0/1853839694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8a8006da60 0x7f8a800b3360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009870 tx=0x7f8a6802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a1f173455dda3a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:34.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.659+0000 7f8a7ea00640 1 -- 172.21.15.40:0/1853839694 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68035020 con 0x7f8a8006da60 2024-03-08T06:06:34.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.660+0000 7f8a7ea00640 1 -- 172.21.15.40:0/1853839694 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a68031640 con 0x7f8a8006da60 2024-03-08T06:06:34.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.660+0000 7f8a7ea00640 1 -- 172.21.15.40:0/1853839694 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6802d040 con 0x7f8a8006da60 2024-03-08T06:06:34.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.660+0000 7f8a85200640 1 -- 172.21.15.40:0/1853839694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8a8006da60 msgr2=0x7f8a800b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.660+0000 7f8a85200640 1 --2- 172.21.15.40:0/1853839694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8a8006da60 0x7f8a800b3360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009870 tx=0x7f8a6802b8f0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.660+0000 7f8a85200640 1 -- 172.21.15.40:0/1853839694 shutdown_connections 2024-03-08T06:06:34.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.660+0000 7f8a85200640 1 --2- 172.21.15.40:0/1853839694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8a8006da60 0x7f8a800b3360 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.660+0000 7f8a85200640 1 --2- 172.21.15.40:0/1853839694 >> v2:172.21.15.53:3300/0 conn(0x7f8a8014b4b0 0x7f8a8006d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.660+0000 7f8a85200640 1 -- 172.21.15.40:0/1853839694 >> 172.21.15.40:0/1853839694 conn(0x7f8a8006ac50 msgr2=0x7f8a8006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:34.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.660+0000 7f8a85200640 1 -- 172.21.15.40:0/1853839694 shutdown_connections 2024-03-08T06:06:34.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.660+0000 7f8a85200640 1 -- 172.21.15.40:0/1853839694 wait complete. 2024-03-08T06:06:34.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.662+0000 7f8a85200640 1 Processor -- start 2024-03-08T06:06:34.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.662+0000 7f8a85200640 1 -- start start 2024-03-08T06:06:34.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.663+0000 7f8a85200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8a8006da60 0x7f8a80145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.663+0000 7f8a85200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a800b38a0 0x7f8a801f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.663+0000 7f8a85200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a8014b4b0 0x7f8a801ed080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.663+0000 7f8a85200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8014b890 con 0x7f8a8014b4b0 2024-03-08T06:06:34.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.663+0000 7f8a85200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8014be30 con 0x7f8a800b38a0 2024-03-08T06:06:34.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.663+0000 7f8a85200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a801ebfe0 con 0x7f8a8006da60 2024-03-08T06:06:34.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.664+0000 7f8a7fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a8014b4b0 0x7f8a801ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.664+0000 7f8a7fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a8014b4b0 0x7f8a801ed080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36324/0 (socket says 172.21.15.40:36324) 2024-03-08T06:06:34.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.664+0000 7f8a7fe00640 1 -- 172.21.15.40:0/4233019407 learned_addr learned my addr 172.21.15.40:0/4233019407 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:34.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.664+0000 7f8a7f400640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8a8006da60 0x7f8a80145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.664+0000 7f8a76a00640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a800b38a0 0x7f8a801f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.664+0000 7f8a7fe00640 1 -- 172.21.15.40:0/4233019407 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8a8006da60 msgr2=0x7f8a80145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.664+0000 7f8a7fe00640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8a8006da60 0x7f8a80145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.664+0000 7f8a7fe00640 1 -- 172.21.15.40:0/4233019407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a800b38a0 msgr2=0x7f8a801f8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.664+0000 7f8a7fe00640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a800b38a0 0x7f8a801f8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.664+0000 7f8a7fe00640 1 -- 172.21.15.40:0/4233019407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a801ec2c0 con 0x7f8a8014b4b0 2024-03-08T06:06:34.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.665+0000 7f8a7fe00640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a8014b4b0 0x7f8a801ed080 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800c780 tx=0x7f8a7800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:34.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.665+0000 7f8a77e00640 1 -- 172.21.15.40:0/4233019407 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78017070 con 0x7f8a8014b4b0 2024-03-08T06:06:34.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.665+0000 7f8a77e00640 1 -- 172.21.15.40:0/4233019407 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7800a640 con 0x7f8a8014b4b0 2024-03-08T06:06:34.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.665+0000 7f8a77e00640 1 -- 172.21.15.40:0/4233019407 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800aa30 con 0x7f8a8014b4b0 2024-03-08T06:06:34.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.665+0000 7f8a85200640 1 -- 172.21.15.40:0/4233019407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a801ec5a0 con 0x7f8a8014b4b0 2024-03-08T06:06:34.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.665+0000 7f8a85200640 1 -- 172.21.15.40:0/4233019407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a801ecb60 con 0x7f8a8014b4b0 2024-03-08T06:06:34.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.666+0000 7f8a85200640 1 -- 172.21.15.40:0/4233019407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a80075090 con 0x7f8a8014b4b0 2024-03-08T06:06:34.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.667+0000 7f8a77e00640 1 -- 172.21.15.40:0/4233019407 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a7801a070 con 0x7f8a8014b4b0 2024-03-08T06:06:34.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.668+0000 7f8a77e00640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8a5c06b4d0 0x7f8a5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.668+0000 7f8a77e00640 1 -- 172.21.15.40:0/4233019407 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a78089430 con 0x7f8a8014b4b0 2024-03-08T06:06:34.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.668+0000 7f8a7f400640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8a5c06b4d0 0x7f8a5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.668+0000 7f8a77e00640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] conn(0x7f8a5c071390 0x7f8a5c0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.668+0000 7f8a77e00640 1 -- 172.21.15.40:0/4233019407 --> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a5c073e60 con 0x7f8a5c071390 2024-03-08T06:06:34.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.668+0000 7f8a77e00640 1 -- 172.21.15.40:0/4233019407 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a78090050 con 0x7f8a8014b4b0 2024-03-08T06:06:34.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.669+0000 7f8a76a00640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] conn(0x7f8a5c071390 0x7f8a5c0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.669+0000 7f8a7f400640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8a5c06b4d0 0x7f8a5c06d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009a30 tx=0x7f8a70008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:34.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.669+0000 7f8a76a00640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] conn(0x7f8a5c071390 0x7f8a5c0737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:34.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.670+0000 7f8a77e00640 1 -- 172.21.15.40:0/4233019407 <== osd.2 v2:172.21.15.40:6819/1467838663 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8a5c073e60 con 0x7f8a5c071390 2024-03-08T06:06:34.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.685+0000 7f8a85200640 1 -- 172.21.15.40:0/4233019407 --> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8a801470b0 con 0x7f8a5c071390 2024-03-08T06:06:34.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.686+0000 7f8a77e00640 1 -- 172.21.15.40:0/4233019407 <== osd.2 v2:172.21.15.40:6819/1467838663 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8a801470b0 con 0x7f8a5c071390 2024-03-08T06:06:34.686 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:06:34.686 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:06:34.686 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:06:34.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.686+0000 7f8a85200640 1 -- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] conn(0x7f8a5c071390 msgr2=0x7f8a5c0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.686+0000 7f8a85200640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:6819/1467838663,v1:172.21.15.40:6822/1467838663] conn(0x7f8a5c071390 0x7f8a5c0737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.687+0000 7f8a85200640 1 -- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8a5c06b4d0 msgr2=0x7f8a5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.687+0000 7f8a85200640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8a5c06b4d0 0x7f8a5c06d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009a30 tx=0x7f8a70008040 comp rx=0 tx=0).stop 2024-03-08T06:06:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.687+0000 7f8a85200640 1 -- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a8014b4b0 msgr2=0x7f8a801ed080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.687+0000 7f8a85200640 1 --2- 172.21.15.40:0/4233019407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a8014b4b0 0x7f8a801ed080 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800c780 tx=0x7f8a7800cc40 comp rx=0 tx=0).stop 2024-03-08T06:06:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.687+0000 7f8a7fe00640 1 -- 172.21.15.40:0/4233019407 reap_dead start 2024-03-08T06:06:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.687+0000 7f8a85200640 1 -- 172.21.15.40:0/4233019407 shutdown_connections 2024-03-08T06:06:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.687+0000 7f8a85200640 1 -- 172.21.15.40:0/4233019407 >> 172.21.15.40:0/4233019407 conn(0x7f8a8006ac50 msgr2=0x7f8a80148aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.687+0000 7f8a85200640 1 -- 172.21.15.40:0/4233019407 shutdown_connections 2024-03-08T06:06:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.687+0000 7f8a85200640 1 -- 172.21.15.40:0/4233019407 wait complete. 2024-03-08T06:06:34.706 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-08T06:06:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.928+0000 7f5e03e00640 1 Processor -- start 2024-03-08T06:06:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.928+0000 7f5e03e00640 1 -- start start 2024-03-08T06:06:34.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.930+0000 7f5e03e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5e04071b10 0x7f5e04166750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.930+0000 7f5e03e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0406d600 con 0x7f5e041451b0 2024-03-08T06:06:34.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.930+0000 7f5e03e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0406d8e0 con 0x7f5e0406f050 2024-03-08T06:06:34.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.930+0000 7f5e03e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0406dbc0 con 0x7f5e04071b10 2024-03-08T06:06:34.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.930+0000 7f5e02a00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5e041451b0 0x7f5e0415ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36484/0 (socket says 172.21.15.40:36484) 2024-03-08T06:06:34.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.930+0000 7f5e02a00640 1 -- 172.21.15.40:0/3764307884 learned_addr learned my addr 172.21.15.40:0/3764307884 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:34.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.930+0000 7f5e03400640 1 --2- 172.21.15.40:0/3764307884 >> v2:172.21.15.53:3300/0 conn(0x7f5e04071b10 0x7f5e04166750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.931+0000 7f5e01600640 1 -- 172.21.15.40:0/3764307884 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361096822 0 0) 0x7f5e0406d600 con 0x7f5e041451b0 2024-03-08T06:06:34.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.931+0000 7f5e01600640 1 -- 172.21.15.40:0/3764307884 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de8003520 con 0x7f5e041451b0 2024-03-08T06:06:34.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.931+0000 7f5e03400640 1 -- 172.21.15.40:0/3764307884 >> v1:172.21.15.53:6789/0 conn(0x7f5e0406f050 legacy=0x7f5e04071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.931+0000 7f5e03400640 1 -- 172.21.15.40:0/3764307884 >> v1:172.21.15.40:6789/0 conn(0x7f5e041451b0 legacy=0x7f5e0415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.931+0000 7f5e03400640 1 -- 172.21.15.40:0/3764307884 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0406dea0 con 0x7f5e04071b10 2024-03-08T06:06:34.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.933+0000 7f5e03400640 1 --2- 172.21.15.40:0/3764307884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e04071b10 0x7f5e04166750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009d90 tx=0x7f5dfc02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e92107814a8da88 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.933+0000 7f5e01600640 1 -- 172.21.15.40:0/3764307884 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc039070 con 0x7f5e04071b10 2024-03-08T06:06:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.933+0000 7f5e01600640 1 -- 172.21.15.40:0/3764307884 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc02cd00 con 0x7f5e04071b10 2024-03-08T06:06:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.933+0000 7f5e01600640 1 -- 172.21.15.40:0/3764307884 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc034660 con 0x7f5e04071b10 2024-03-08T06:06:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.934+0000 7f5e03e00640 1 -- 172.21.15.40:0/3764307884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e04071b10 msgr2=0x7f5e04166750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.934+0000 7f5e03e00640 1 --2- 172.21.15.40:0/3764307884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e04071b10 0x7f5e04166750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009d90 tx=0x7f5dfc02bf50 comp rx=0 tx=0).stop 2024-03-08T06:06:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.934+0000 7f5e03e00640 1 -- 172.21.15.40:0/3764307884 shutdown_connections 2024-03-08T06:06:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.934+0000 7f5e03e00640 1 --2- 172.21.15.40:0/3764307884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e04071b10 0x7f5e04166750 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.934+0000 7f5e03e00640 1 -- 172.21.15.40:0/3764307884 >> 172.21.15.40:0/3764307884 conn(0x7f5e040b1590 msgr2=0x7f5e040b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.934+0000 7f5e03e00640 1 -- 172.21.15.40:0/3764307884 shutdown_connections 2024-03-08T06:06:34.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.934+0000 7f5e03e00640 1 -- 172.21.15.40:0/3764307884 wait complete. 2024-03-08T06:06:34.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.936+0000 7f5e03e00640 1 Processor -- start 2024-03-08T06:06:34.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.936+0000 7f5e03e00640 1 -- start start 2024-03-08T06:06:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e03e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0406f050 0x7f5e041e5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e03e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e04071b10 0x7f5e040b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e03e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e041451b0 0x7f5e040b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e03e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0415e330 con 0x7f5e0406f050 2024-03-08T06:06:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e03e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0415e1b0 con 0x7f5e04071b10 2024-03-08T06:06:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e03e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0415e4b0 con 0x7f5e041451b0 2024-03-08T06:06:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e02a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0406f050 0x7f5e041e5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e02a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0406f050 0x7f5e041e5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36328/0 (socket says 172.21.15.40:36328) 2024-03-08T06:06:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e02a00640 1 -- 172.21.15.40:0/4005895513 learned_addr learned my addr 172.21.15.40:0/4005895513 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e02a00640 1 -- 172.21.15.40:0/4005895513 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e041451b0 msgr2=0x7f5e040b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e02a00640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e041451b0 0x7f5e040b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e02a00640 1 -- 172.21.15.40:0/4005895513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e04071b10 msgr2=0x7f5e040b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e02000640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e04071b10 0x7f5e040b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e02a00640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e04071b10 0x7f5e040b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.937+0000 7f5e02a00640 1 -- 172.21.15.40:0/4005895513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e04143210 con 0x7f5e0406f050 2024-03-08T06:06:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.938+0000 7f5e02a00640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0406f050 0x7f5e041e5c20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c3b0 tx=0x7f5df400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.938+0000 7f5dfaa00640 1 -- 172.21.15.40:0/4005895513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4017070 con 0x7f5e0406f050 2024-03-08T06:06:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.938+0000 7f5dfaa00640 1 -- 172.21.15.40:0/4005895513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df4005ce0 con 0x7f5e0406f050 2024-03-08T06:06:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.938+0000 7f5dfaa00640 1 -- 172.21.15.40:0/4005895513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4007310 con 0x7f5e0406f050 2024-03-08T06:06:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.938+0000 7f5e03e00640 1 -- 172.21.15.40:0/4005895513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e041e6160 con 0x7f5e0406f050 2024-03-08T06:06:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.938+0000 7f5e03e00640 1 -- 172.21.15.40:0/4005895513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e041e6d00 con 0x7f5e0406f050 2024-03-08T06:06:34.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.939+0000 7f5e03e00640 1 -- 172.21.15.40:0/4005895513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e04075090 con 0x7f5e0406f050 2024-03-08T06:06:34.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.940+0000 7f5dfaa00640 1 -- 172.21.15.40:0/4005895513 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5df401a070 con 0x7f5e0406f050 2024-03-08T06:06:34.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.941+0000 7f5dfaa00640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5de406b520 0x7f5de406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:34.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.941+0000 7f5dfaa00640 1 -- 172.21.15.40:0/4005895513 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5df4089390 con 0x7f5e0406f050 2024-03-08T06:06:34.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.942+0000 7f5e02000640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5de406b520 0x7f5de406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:34.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.942+0000 7f5e02000640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5de406b520 0x7f5de406d9e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5e04160660 tx=0x7f5df0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:34.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:34.946+0000 7f5dfaa00640 1 -- 172.21.15.40:0/4005895513 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e04143210 con 0x7f5e0406f050 2024-03-08T06:06:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.144+0000 7f5e03e00640 1 -- 172.21.15.40:0/4005895513 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e040b1bc0 con 0x7f5de406b520 2024-03-08T06:06:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.145+0000 7f5dfaa00640 1 -- 172.21.15.40:0/4005895513 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5e040b1bc0 con 0x7f5de406b520 2024-03-08T06:06:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2024-03-08T06:06:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.148+0000 7f5e03e00640 1 -- 172.21.15.40:0/4005895513 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5de406b520 msgr2=0x7f5de406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.148+0000 7f5e03e00640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5de406b520 0x7f5de406d9e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5e04160660 tx=0x7f5df0009040 comp rx=0 tx=0).stop 2024-03-08T06:06:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.148+0000 7f5e03e00640 1 -- 172.21.15.40:0/4005895513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0406f050 msgr2=0x7f5e041e5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.148+0000 7f5e03e00640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0406f050 0x7f5e041e5c20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c3b0 tx=0x7f5df400c870 comp rx=0 tx=0).stop 2024-03-08T06:06:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.148+0000 7f5e03e00640 1 -- 172.21.15.40:0/4005895513 shutdown_connections 2024-03-08T06:06:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.148+0000 7f5e03e00640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5de406b520 0x7f5de406d9e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.148+0000 7f5e03e00640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e041451b0 0x7f5e040b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.148+0000 7f5e03e00640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e04071b10 0x7f5e040b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.148+0000 7f5e03e00640 1 --2- 172.21.15.40:0/4005895513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e0406f050 0x7f5e041e5c20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.148+0000 7f5e03e00640 1 -- 172.21.15.40:0/4005895513 >> 172.21.15.40:0/4005895513 conn(0x7f5e040b1590 msgr2=0x7f5e040b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.149+0000 7f5e03e00640 1 -- 172.21.15.40:0/4005895513 shutdown_connections 2024-03-08T06:06:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.149+0000 7f5e03e00640 1 -- 172.21.15.40:0/4005895513 wait complete. 2024-03-08T06:06:35.168 INFO:tasks.ceph:Scrubbing osd.3 2024-03-08T06:06:35.168 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:06:35.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.395+0000 7fc0fe200640 1 Processor -- start 2024-03-08T06:06:35.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.395+0000 7fc0fe200640 1 -- start start 2024-03-08T06:06:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.397+0000 7fc0fe200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc0f8144540 0x7fc0f806f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.397+0000 7fc0fe200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc0f80b3d60 0x7fc0f80b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.397+0000 7fc0fe200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f806d850 con 0x7fc0f80717f0 2024-03-08T06:06:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.397+0000 7fc0fe200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f806e700 con 0x7fc0f80b3d60 2024-03-08T06:06:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.397+0000 7fc0fe200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f80b4700 con 0x7fc0f8144540 2024-03-08T06:06:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.397+0000 7fc0f6a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc0f80b3d60 0x7fc0f80b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.397+0000 7fc0f7400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc0f8144540 0x7fc0f806f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.398+0000 7fc0f7400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc0f8144540 0x7fc0f806f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:54714/0 (socket says 172.21.15.40:54714) 2024-03-08T06:06:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.398+0000 7fc0f7400640 1 -- 172.21.15.40:0/3189477381 learned_addr learned my addr 172.21.15.40:0/3189477381 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.398+0000 7fc0f6a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc0f80b3d60 0x7fc0f80b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:49766/0 (socket says 172.21.15.40:49766) 2024-03-08T06:06:35.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.398+0000 7fc0f7400640 1 -- 172.21.15.40:0/3189477381 >> v2:172.21.15.53:3300/0 conn(0x7fc0f80b3d60 msgr2=0x7fc0f80b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:35.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.398+0000 7fc0f7400640 1 --2- 172.21.15.40:0/3189477381 >> v2:172.21.15.53:3300/0 conn(0x7fc0f80b3d60 0x7fc0f80b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.398+0000 7fc0f7400640 1 -- 172.21.15.40:0/3189477381 >> v1:172.21.15.53:6789/0 conn(0x7fc0f80717f0 legacy=0x7fc0f8071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.398+0000 7fc0f7400640 1 -- 172.21.15.40:0/3189477381 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f806daf0 con 0x7fc0f8144540 2024-03-08T06:06:35.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.398+0000 7fc0f6a00640 1 --2- 172.21.15.40:0/3189477381 >> v2:172.21.15.53:3300/0 conn(0x7fc0f80b3d60 0x7fc0f80b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:06:35.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.400+0000 7fc0f7400640 1 --2- 172.21.15.40:0/3189477381 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc0f8144540 0x7fc0f806f4d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009870 tx=0x7fc0e802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c07ae3e78628ad58 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:35.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.400+0000 7fc0f6000640 1 -- 172.21.15.40:0/3189477381 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8034070 con 0x7fc0f8144540 2024-03-08T06:06:35.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.400+0000 7fc0f6000640 1 -- 172.21.15.40:0/3189477381 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e8030840 con 0x7fc0f8144540 2024-03-08T06:06:35.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.400+0000 7fc0f6000640 1 -- 172.21.15.40:0/3189477381 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8030c10 con 0x7fc0f8144540 2024-03-08T06:06:35.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.401+0000 7fc0fe200640 1 -- 172.21.15.40:0/3189477381 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc0f8144540 msgr2=0x7fc0f806f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.401+0000 7fc0fe200640 1 --2- 172.21.15.40:0/3189477381 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc0f8144540 0x7fc0f806f4d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009870 tx=0x7fc0e802b980 comp rx=0 tx=0).stop 2024-03-08T06:06:35.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.401+0000 7fc0fe200640 1 -- 172.21.15.40:0/3189477381 shutdown_connections 2024-03-08T06:06:35.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.401+0000 7fc0fe200640 1 --2- 172.21.15.40:0/3189477381 >> v2:172.21.15.53:3300/0 conn(0x7fc0f80b3d60 0x7fc0f80b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.401+0000 7fc0fe200640 1 --2- 172.21.15.40:0/3189477381 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc0f8144540 0x7fc0f806f4d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.401+0000 7fc0fe200640 1 -- 172.21.15.40:0/3189477381 >> 172.21.15.40:0/3189477381 conn(0x7fc0f80cfaa0 msgr2=0x7fc0f80cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:35.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.401+0000 7fc0fe200640 1 -- 172.21.15.40:0/3189477381 shutdown_connections 2024-03-08T06:06:35.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.401+0000 7fc0fe200640 1 -- 172.21.15.40:0/3189477381 wait complete. 2024-03-08T06:06:35.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.403+0000 7fc0fe200640 1 Processor -- start 2024-03-08T06:06:35.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.403+0000 7fc0fe200640 1 -- start start 2024-03-08T06:06:35.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.404+0000 7fc0fe200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0f80717f0 0x7fc0f815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.404+0000 7fc0fe200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc0f80b3d60 0x7fc0f80addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.404+0000 7fc0fe200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f8144540 0x7fc0f814fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.404+0000 7fc0fe200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f806e390 con 0x7fc0f8144540 2024-03-08T06:06:35.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.404+0000 7fc0fe200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f806e0b0 con 0x7fc0f80717f0 2024-03-08T06:06:35.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.404+0000 7fc0fe200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f815f620 con 0x7fc0f80b3d60 2024-03-08T06:06:35.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.405+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f8144540 0x7fc0f814fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.405+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f8144540 0x7fc0f814fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36338/0 (socket says 172.21.15.40:36338) 2024-03-08T06:06:35.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.405+0000 7fc0f7e00640 1 -- 172.21.15.40:0/3801466789 learned_addr learned my addr 172.21.15.40:0/3801466789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.405+0000 7fc0f6a00640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc0f80b3d60 0x7fc0f80addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.405+0000 7fc0f7400640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0f80717f0 0x7fc0f815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.405+0000 7fc0f7e00640 1 -- 172.21.15.40:0/3801466789 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc0f80b3d60 msgr2=0x7fc0f80addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.405+0000 7fc0f7e00640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc0f80b3d60 0x7fc0f80addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.405+0000 7fc0f7e00640 1 -- 172.21.15.40:0/3801466789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0f80717f0 msgr2=0x7fc0f815db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.405+0000 7fc0f7e00640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0f80717f0 0x7fc0f815db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.405+0000 7fc0f7e00640 1 -- 172.21.15.40:0/3801466789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f80acd10 con 0x7fc0f8144540 2024-03-08T06:06:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.406+0000 7fc0f7e00640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f8144540 0x7fc0f814fdb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0001eb0 tx=0x7fc0f000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.406+0000 7fc0ef400640 1 -- 172.21.15.40:0/3801466789 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0017070 con 0x7fc0f8144540 2024-03-08T06:06:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.406+0000 7fc0ef400640 1 -- 172.21.15.40:0/3801466789 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f00052d0 con 0x7fc0f8144540 2024-03-08T06:06:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.406+0000 7fc0fe200640 1 -- 172.21.15.40:0/3801466789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f80acff0 con 0x7fc0f8144540 2024-03-08T06:06:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.406+0000 7fc0ef400640 1 -- 172.21.15.40:0/3801466789 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0003810 con 0x7fc0f8144540 2024-03-08T06:06:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.406+0000 7fc0fe200640 1 -- 172.21.15.40:0/3801466789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f80ad5b0 con 0x7fc0f8144540 2024-03-08T06:06:35.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.407+0000 7fc0fe200640 1 -- 172.21.15.40:0/3801466789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0f8067cb0 con 0x7fc0f8144540 2024-03-08T06:06:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.408+0000 7fc0ef400640 1 -- 172.21.15.40:0/3801466789 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0f001a070 con 0x7fc0f8144540 2024-03-08T06:06:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.409+0000 7fc0ef400640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc0d406b4d0 0x7fc0d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.409+0000 7fc0ef400640 1 -- 172.21.15.40:0/3801466789 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc0f0089320 con 0x7fc0f8144540 2024-03-08T06:06:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.409+0000 7fc0ef400640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] conn(0x7fc0d4071390 0x7fc0d40737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.410+0000 7fc0ef400640 1 -- 172.21.15.40:0/3801466789 --> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0d4073e60 con 0x7fc0d4071390 2024-03-08T06:06:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.410+0000 7fc0ef400640 1 -- 172.21.15.40:0/3801466789 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0f0090050 con 0x7fc0f8144540 2024-03-08T06:06:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.410+0000 7fc0f7400640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc0d406b4d0 0x7fc0d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.410+0000 7fc0f6a00640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] conn(0x7fc0d4071390 0x7fc0d40737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.410+0000 7fc0f7400640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc0d406b4d0 0x7fc0d406d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8036000 tx=0x7fc0e803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.411+0000 7fc0f6a00640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] conn(0x7fc0d4071390 0x7fc0d40737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:35.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.411+0000 7fc0ef400640 1 -- 172.21.15.40:0/3801466789 <== osd.3 v2:172.21.15.40:6815/405718921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc0d4073e60 con 0x7fc0d4071390 2024-03-08T06:06:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.426+0000 7fc0fe200640 1 -- 172.21.15.40:0/3801466789 --> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc0f80d08a0 con 0x7fc0d4071390 2024-03-08T06:06:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.427+0000 7fc0ef400640 1 -- 172.21.15.40:0/3801466789 <== osd.3 v2:172.21.15.40:6815/405718921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc0f80d08a0 con 0x7fc0d4071390 2024-03-08T06:06:35.427 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:06:35.427 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:06:35.427 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:06:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.427+0000 7fc0fe200640 1 -- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] conn(0x7fc0d4071390 msgr2=0x7fc0d40737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.427+0000 7fc0fe200640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:6815/405718921,v1:172.21.15.40:6817/405718921] conn(0x7fc0d4071390 0x7fc0d40737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.427+0000 7fc0fe200640 1 -- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc0d406b4d0 msgr2=0x7fc0d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.427+0000 7fc0fe200640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fc0d406b4d0 0x7fc0d406d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8036000 tx=0x7fc0e803b040 comp rx=0 tx=0).stop 2024-03-08T06:06:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.427+0000 7fc0fe200640 1 -- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f8144540 msgr2=0x7fc0f814fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.427+0000 7fc0fe200640 1 --2- 172.21.15.40:0/3801466789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f8144540 0x7fc0f814fdb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0001eb0 tx=0x7fc0f000c890 comp rx=0 tx=0).stop 2024-03-08T06:06:35.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.427+0000 7fc0f7e00640 1 -- 172.21.15.40:0/3801466789 reap_dead start 2024-03-08T06:06:35.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.427+0000 7fc0fe200640 1 -- 172.21.15.40:0/3801466789 shutdown_connections 2024-03-08T06:06:35.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.427+0000 7fc0fe200640 1 -- 172.21.15.40:0/3801466789 >> 172.21.15.40:0/3801466789 conn(0x7fc0f80cfaa0 msgr2=0x7fc0f806ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:35.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.428+0000 7fc0fe200640 1 -- 172.21.15.40:0/3801466789 shutdown_connections 2024-03-08T06:06:35.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.428+0000 7fc0fe200640 1 -- 172.21.15.40:0/3801466789 wait complete. 2024-03-08T06:06:35.448 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-08T06:06:35.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.672+0000 7fa7c1200640 1 Processor -- start 2024-03-08T06:06:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.673+0000 7fa7c1200640 1 -- start start 2024-03-08T06:06:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.674+0000 7fa7c1200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7bc0cd730 0x7fa7bc0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.674+0000 7fa7c1200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0cc690 con 0x7fa7bc0717d0 2024-03-08T06:06:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.674+0000 7fa7c1200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0cc970 con 0x7fa7bc145250 2024-03-08T06:06:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.674+0000 7fa7c1200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7bc0ccc50 con 0x7fa7bc0cd730 2024-03-08T06:06:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.675+0000 7fa7b1400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7bc0cd730 0x7fa7bc0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.675+0000 7fa7b1400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7bc0cd730 0x7fa7bc0cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36350/0 (socket says 172.21.15.40:36350) 2024-03-08T06:06:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.675+0000 7fa7b1400640 1 -- 172.21.15.40:0/4002729844 learned_addr learned my addr 172.21.15.40:0/4002729844 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.675+0000 7fa7b1400640 1 -- 172.21.15.40:0/4002729844 >> v1:172.21.15.53:6789/0 conn(0x7fa7bc145250 legacy=0x7fa7bc06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.675+0000 7fa7b1400640 1 -- 172.21.15.40:0/4002729844 >> v1:172.21.15.40:6789/0 conn(0x7fa7bc0717d0 legacy=0x7fa7bc15c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.675+0000 7fa7b1400640 1 -- 172.21.15.40:0/4002729844 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc0ccf30 con 0x7fa7bc0cd730 2024-03-08T06:06:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.675+0000 7fa7b9400640 1 -- 172.21.15.40:0/4002729844 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019772329 0 0) 0x7fa7bc0cc690 con 0x7fa7bc0717d0 2024-03-08T06:06:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.677+0000 7fa7b1400640 1 --2- 172.21.15.40:0/4002729844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0cd730 0x7fa7bc0cc0b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009870 tx=0x7fa7a402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204fd65db1c673f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.677+0000 7fa7b9400640 1 -- 172.21.15.40:0/4002729844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a402f840 con 0x7fa7bc0cd730 2024-03-08T06:06:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.677+0000 7fa7b9400640 1 -- 172.21.15.40:0/4002729844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a402f9e0 con 0x7fa7bc0cd730 2024-03-08T06:06:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.677+0000 7fa7b9400640 1 -- 172.21.15.40:0/4002729844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a402fdb0 con 0x7fa7bc0cd730 2024-03-08T06:06:35.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.677+0000 7fa7c1200640 1 -- 172.21.15.40:0/4002729844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0cd730 msgr2=0x7fa7bc0cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.677+0000 7fa7c1200640 1 --2- 172.21.15.40:0/4002729844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0cd730 0x7fa7bc0cc0b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009870 tx=0x7fa7a402ba20 comp rx=0 tx=0).stop 2024-03-08T06:06:35.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.678+0000 7fa7c1200640 1 -- 172.21.15.40:0/4002729844 shutdown_connections 2024-03-08T06:06:35.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.678+0000 7fa7c1200640 1 --2- 172.21.15.40:0/4002729844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0cd730 0x7fa7bc0cc0b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.678+0000 7fa7c1200640 1 -- 172.21.15.40:0/4002729844 >> 172.21.15.40:0/4002729844 conn(0x7fa7bc06ae30 msgr2=0x7fa7bc06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:35.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.678+0000 7fa7c1200640 1 -- 172.21.15.40:0/4002729844 shutdown_connections 2024-03-08T06:06:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.678+0000 7fa7c1200640 1 -- 172.21.15.40:0/4002729844 wait complete. 2024-03-08T06:06:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.680+0000 7fa7c1200640 1 Processor -- start 2024-03-08T06:06:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.680+0000 7fa7c1200640 1 -- start start 2024-03-08T06:06:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7c1200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7bc0717d0 0x7fa7bc13eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7c1200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc145250 0x7fa7bc15a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7c1200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7bc15acd0 0x7fa7bc131400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7c1200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc1457b0 con 0x7fa7bc145250 2024-03-08T06:06:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7c1200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc145630 con 0x7fa7bc15acd0 2024-03-08T06:06:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7c1200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc071c40 con 0x7fa7bc0717d0 2024-03-08T06:06:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7b1400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc145250 0x7fa7bc15a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7b1400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc145250 0x7fa7bc15a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36352/0 (socket says 172.21.15.40:36352) 2024-03-08T06:06:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7b1400640 1 -- 172.21.15.40:0/2937824686 learned_addr learned my addr 172.21.15.40:0/2937824686 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7b9e00640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7bc0717d0 0x7fa7bc13eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7b1400640 1 -- 172.21.15.40:0/2937824686 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7bc0717d0 msgr2=0x7fa7bc13eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.682+0000 7fa7b1400640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7bc0717d0 0x7fa7bc13eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.681+0000 7fa7ba800640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7bc15acd0 0x7fa7bc131400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.682+0000 7fa7b1400640 1 -- 172.21.15.40:0/2937824686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7bc15acd0 msgr2=0x7fa7bc131400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.682+0000 7fa7b1400640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7bc15acd0 0x7fa7bc131400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.682+0000 7fa7b1400640 1 -- 172.21.15.40:0/2937824686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc159cf0 con 0x7fa7bc145250 2024-03-08T06:06:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.682+0000 7fa7b9e00640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7bc0717d0 0x7fa7bc13eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:35.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.682+0000 7fa7ba800640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7bc15acd0 0x7fa7bc131400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:35.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.682+0000 7fa7b1400640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc145250 0x7fa7bc15a790 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009870 tx=0x7fa7a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.682+0000 7fa7b2a00640 1 -- 172.21.15.40:0/2937824686 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4041400 con 0x7fa7bc145250 2024-03-08T06:06:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.682+0000 7fa7b2a00640 1 -- 172.21.15.40:0/2937824686 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a4046070 con 0x7fa7bc145250 2024-03-08T06:06:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.682+0000 7fa7c1200640 1 -- 172.21.15.40:0/2937824686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc131de0 con 0x7fa7bc145250 2024-03-08T06:06:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.682+0000 7fa7b2a00640 1 -- 172.21.15.40:0/2937824686 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4041800 con 0x7fa7bc145250 2024-03-08T06:06:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.683+0000 7fa7c1200640 1 -- 172.21.15.40:0/2937824686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc159f90 con 0x7fa7bc145250 2024-03-08T06:06:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.684+0000 7fa7c1200640 1 -- 172.21.15.40:0/2937824686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc06f870 con 0x7fa7bc145250 2024-03-08T06:06:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.685+0000 7fa7b2a00640 1 -- 172.21.15.40:0/2937824686 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7a4040070 con 0x7fa7bc145250 2024-03-08T06:06:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.686+0000 7fa7b2a00640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa79806b4d0 0x7fa79806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.686+0000 7fa7b2a00640 1 -- 172.21.15.40:0/2937824686 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa7a40ac7f0 con 0x7fa7bc145250 2024-03-08T06:06:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.686+0000 7fa7b9e00640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa79806b4d0 0x7fa79806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.687+0000 7fa7b9e00640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa79806b4d0 0x7fa79806d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000f60 tx=0x7fa7ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:35.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.691+0000 7fa7b2a00640 1 -- 172.21.15.40:0/2937824686 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7bc06f870 con 0x7fa7bc145250 2024-03-08T06:06:35.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.889+0000 7fa7c1200640 1 -- 172.21.15.40:0/2937824686 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7bc067cb0 con 0x7fa79806b4d0 2024-03-08T06:06:35.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.890+0000 7fa7b2a00640 1 -- 172.21.15.40:0/2937824686 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa7bc067cb0 con 0x7fa79806b4d0 2024-03-08T06:06:35.890 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2024-03-08T06:06:35.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.893+0000 7fa7c1200640 1 -- 172.21.15.40:0/2937824686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa79806b4d0 msgr2=0x7fa79806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.893+0000 7fa7c1200640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa79806b4d0 0x7fa79806d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000f60 tx=0x7fa7ac009040 comp rx=0 tx=0).stop 2024-03-08T06:06:35.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.893+0000 7fa7c1200640 1 -- 172.21.15.40:0/2937824686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc145250 msgr2=0x7fa7bc15a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:35.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.893+0000 7fa7c1200640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc145250 0x7fa7bc15a790 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009870 tx=0x7fa7a4004730 comp rx=0 tx=0).stop 2024-03-08T06:06:35.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.893+0000 7fa7c1200640 1 -- 172.21.15.40:0/2937824686 shutdown_connections 2024-03-08T06:06:35.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.893+0000 7fa7c1200640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7fa79806b4d0 0x7fa79806d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.893+0000 7fa7c1200640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7bc15acd0 0x7fa7bc131400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.893+0000 7fa7c1200640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc145250 0x7fa7bc15a790 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.893+0000 7fa7c1200640 1 --2- 172.21.15.40:0/2937824686 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa7bc0717d0 0x7fa7bc13eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:35.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.893+0000 7fa7c1200640 1 -- 172.21.15.40:0/2937824686 >> 172.21.15.40:0/2937824686 conn(0x7fa7bc06ae30 msgr2=0x7fa7bc0afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:35.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.893+0000 7fa7c1200640 1 -- 172.21.15.40:0/2937824686 shutdown_connections 2024-03-08T06:06:35.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:35.894+0000 7fa7c1200640 1 -- 172.21.15.40:0/2937824686 wait complete. 2024-03-08T06:06:35.915 INFO:tasks.ceph:Scrubbing osd.4 2024-03-08T06:06:35.915 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:06:36.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.143+0000 7f8debe00640 1 Processor -- start 2024-03-08T06:06:36.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.143+0000 7f8debe00640 1 -- start start 2024-03-08T06:06:36.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.145+0000 7f8debe00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8dec06e8a0 0x7f8dec0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.145+0000 7f8debe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dec15e5a0 0x7f8dec0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.145+0000 7f8debe00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dec0b37e0 con 0x7f8dec06ec80 2024-03-08T06:06:36.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.145+0000 7f8debe00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dec15ea30 con 0x7f8dec06e8a0 2024-03-08T06:06:36.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.145+0000 7f8debe00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec15ed10 con 0x7f8dec15e5a0 2024-03-08T06:06:36.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.145+0000 7f8deb400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8dec06ec80 0x7f8dec0cce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36506/0 (socket says 172.21.15.40:36506) 2024-03-08T06:06:36.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.145+0000 7f8deb400640 1 -- 172.21.15.40:0/3539939447 learned_addr learned my addr 172.21.15.40:0/3539939447 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:36.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.145+0000 7f8deaa00640 1 --2- 172.21.15.40:0/3539939447 >> v2:172.21.15.53:3300/0 conn(0x7f8dec06e8a0 0x7f8dec0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8dea000640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dec15e5a0 0x7f8dec0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773244132 0 0) 0x7f8dec0b37e0 con 0x7f8dec06ec80 2024-03-08T06:06:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dd0005730 con 0x7f8dec06ec80 2024-03-08T06:06:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3648362925 0 0) 0x7f8dd0005730 con 0x7f8dec06ec80 2024-03-08T06:06:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dec15e5a0 msgr2=0x7f8dec0cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dec15e5a0 0x7f8dec0cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 >> v2:172.21.15.53:3300/0 conn(0x7f8dec06e8a0 msgr2=0x7f8dec0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 --2- 172.21.15.40:0/3539939447 >> v2:172.21.15.53:3300/0 conn(0x7f8dec06e8a0 0x7f8dec0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec15eff0 con 0x7f8dec06ec80 2024-03-08T06:06:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f8de4001690 con 0x7f8dec06ec80 2024-03-08T06:06:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 >> v1:172.21.15.40:6789/0 conn(0x7f8dec06ec80 legacy=0x7f8dec0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dd0005c30 0x7f8dd0006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dd0006a10 0x7f8dd0006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dd0007390 0x7f8dd000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec15ea30 con 0x7f8dd0007390 2024-03-08T06:06:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec15ed10 con 0x7f8dd0005c30 2024-03-08T06:06:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.146+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd000e110 con 0x7f8dd0006a10 2024-03-08T06:06:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.147+0000 7f8deb400640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dd0007390 0x7f8dd000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.147+0000 7f8deaa00640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dd0005c30 0x7f8dd0006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.147+0000 7f8dea000640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dd0006a10 0x7f8dd0006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.147+0000 7f8deb400640 1 -- 172.21.15.40:0/3539939447 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dd0006a10 msgr2=0x7f8dd0006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.147+0000 7f8deb400640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dd0006a10 0x7f8dd0006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.147+0000 7f8deb400640 1 -- 172.21.15.40:0/3539939447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dd0005c30 msgr2=0x7f8dd0006050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.147+0000 7f8deb400640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dd0005c30 0x7f8dd0006050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.147+0000 7f8deb400640 1 -- 172.21.15.40:0/3539939447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd0005730 con 0x7f8dd0007390 2024-03-08T06:06:36.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.148+0000 7f8deb400640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dd0007390 0x7f8dd000dbd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8de4007fd0 tx=0x7f8dd0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0cdc96469be620 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:36.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.148+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4007ca0 con 0x7f8dd0007390 2024-03-08T06:06:36.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.149+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de4030070 con 0x7f8dd0007390 2024-03-08T06:06:36.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.149+0000 7f8de9600640 1 -- 172.21.15.40:0/3539939447 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4005d10 con 0x7f8dd0007390 2024-03-08T06:06:36.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.149+0000 7f8debe00640 1 -- 172.21.15.40:0/3539939447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dd0007390 msgr2=0x7f8dd000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.149+0000 7f8debe00640 1 --2- 172.21.15.40:0/3539939447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dd0007390 0x7f8dd000dbd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8de4007fd0 tx=0x7f8dd0007a40 comp rx=0 tx=0).stop 2024-03-08T06:06:36.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.149+0000 7f8deb400640 1 -- 172.21.15.40:0/3539939447 reap_dead start 2024-03-08T06:06:36.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.149+0000 7f8debe00640 1 -- 172.21.15.40:0/3539939447 shutdown_connections 2024-03-08T06:06:36.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.149+0000 7f8debe00640 1 -- 172.21.15.40:0/3539939447 >> 172.21.15.40:0/3539939447 conn(0x7f8dec06ac50 msgr2=0x7f8dec06b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:36.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.149+0000 7f8debe00640 1 -- 172.21.15.40:0/3539939447 shutdown_connections 2024-03-08T06:06:36.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.149+0000 7f8debe00640 1 -- 172.21.15.40:0/3539939447 wait complete. 2024-03-08T06:06:36.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.151+0000 7f8debe00640 1 Processor -- start 2024-03-08T06:06:36.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.151+0000 7f8debe00640 1 -- start start 2024-03-08T06:06:36.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.152+0000 7f8debe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dd0007390 0x7f8dec1ef740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.152+0000 7f8debe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dec14bcb0 0x7f8dec0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.152+0000 7f8debe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dec14dac0 0x7f8dec0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.152+0000 7f8debe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec0cd6c0 con 0x7f8dec14dac0 2024-03-08T06:06:36.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.152+0000 7f8debe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec0cd540 con 0x7f8dec14bcb0 2024-03-08T06:06:36.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.152+0000 7f8debe00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec0cd840 con 0x7f8dd0007390 2024-03-08T06:06:36.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.152+0000 7f8deb400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dec14dac0 0x7f8dec0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.153+0000 7f8deb400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dec14dac0 0x7f8dec0b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36362/0 (socket says 172.21.15.40:36362) 2024-03-08T06:06:36.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.153+0000 7f8deb400640 1 -- 172.21.15.40:0/2291929669 learned_addr learned my addr 172.21.15.40:0/2291929669 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:36.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.153+0000 7f8dea000640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dec14bcb0 0x7f8dec0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.153+0000 7f8deaa00640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dd0007390 0x7f8dec1ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.153+0000 7f8deb400640 1 -- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dd0007390 msgr2=0x7f8dec1ef740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.153+0000 7f8deb400640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dd0007390 0x7f8dec1ef740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.153+0000 7f8deb400640 1 -- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dec14bcb0 msgr2=0x7f8dec0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.153+0000 7f8deb400640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dec14bcb0 0x7f8dec0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.153+0000 7f8deb400640 1 -- 172.21.15.40:0/2291929669 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec14dfc0 con 0x7f8dec14dac0 2024-03-08T06:06:36.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.153+0000 7f8deaa00640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8dd0007390 0x7f8dec1ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:36.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.153+0000 7f8deb400640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dec14dac0 0x7f8dec0b4930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8de402c450 tx=0x7f8de4001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:36.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.154+0000 7f8de2a00640 1 -- 172.21.15.40:0/2291929669 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4001be0 con 0x7f8dec14dac0 2024-03-08T06:06:36.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.154+0000 7f8de2a00640 1 -- 172.21.15.40:0/2291929669 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de40047e0 con 0x7f8dec14dac0 2024-03-08T06:06:36.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.154+0000 7f8de2a00640 1 -- 172.21.15.40:0/2291929669 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4004b90 con 0x7f8dec14dac0 2024-03-08T06:06:36.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.154+0000 7f8debe00640 1 -- 172.21.15.40:0/2291929669 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dec14e2a0 con 0x7f8dec14dac0 2024-03-08T06:06:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.154+0000 7f8debe00640 1 -- 172.21.15.40:0/2291929669 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dec14e860 con 0x7f8dec14dac0 2024-03-08T06:06:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.155+0000 7f8debe00640 1 -- 172.21.15.40:0/2291929669 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8dec14ec40 con 0x7f8dec14dac0 2024-03-08T06:06:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.156+0000 7f8de2a00640 1 -- 172.21.15.40:0/2291929669 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8de403e030 con 0x7f8dec14dac0 2024-03-08T06:06:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.157+0000 7f8de2a00640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8dc806b4d0 0x7f8dc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.157+0000 7f8de2a00640 1 -- 172.21.15.40:0/2291929669 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8de40aa650 con 0x7f8dec14dac0 2024-03-08T06:06:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.157+0000 7f8de2a00640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] conn(0x7f8dc8071390 0x7f8dc80737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.157+0000 7f8deaa00640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8dc806b4d0 0x7f8dc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.157+0000 7f8de2a00640 1 -- 172.21.15.40:0/2291929669 --> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8de403ae50 con 0x7f8dc8071390 2024-03-08T06:06:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.157+0000 7f8de2a00640 1 -- 172.21.15.40:0/2291929669 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8de40b1050 con 0x7f8dec14dac0 2024-03-08T06:06:36.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.158+0000 7f8dea000640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] conn(0x7f8dc8071390 0x7f8dc80737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.158+0000 7f8deaa00640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8dc806b4d0 0x7f8dc806d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc007440 tx=0x7f8ddc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:36.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.159+0000 7f8dea000640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] conn(0x7f8dc8071390 0x7f8dc80737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:36.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.161+0000 7f8de2a00640 1 -- 172.21.15.40:0/2291929669 <== osd.4 v2:172.21.15.53:6816/2858249123 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8de403ae50 con 0x7f8dc8071390 2024-03-08T06:06:36.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.175+0000 7f8debe00640 1 -- 172.21.15.40:0/2291929669 --> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8dec075090 con 0x7f8dc8071390 2024-03-08T06:06:36.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.176+0000 7f8de2a00640 1 -- 172.21.15.40:0/2291929669 <== osd.4 v2:172.21.15.53:6816/2858249123 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8dec075090 con 0x7f8dc8071390 2024-03-08T06:06:36.177 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:06:36.177 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:06:36.177 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:06:36.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.177+0000 7f8debe00640 1 -- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] conn(0x7f8dc8071390 msgr2=0x7f8dc80737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.177+0000 7f8debe00640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.53:6816/2858249123,v1:172.21.15.53:6817/2858249123] conn(0x7f8dc8071390 0x7f8dc80737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.177+0000 7f8debe00640 1 -- 172.21.15.40:0/2291929669 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8dc806b4d0 msgr2=0x7f8dc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.177+0000 7f8debe00640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8dc806b4d0 0x7f8dc806d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc007440 tx=0x7f8ddc00b040 comp rx=0 tx=0).stop 2024-03-08T06:06:36.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.177+0000 7f8debe00640 1 -- 172.21.15.40:0/2291929669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dec14dac0 msgr2=0x7f8dec0b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.177+0000 7f8debe00640 1 --2- 172.21.15.40:0/2291929669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dec14dac0 0x7f8dec0b4930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8de402c450 tx=0x7f8de4001990 comp rx=0 tx=0).stop 2024-03-08T06:06:36.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.177+0000 7f8deb400640 1 -- 172.21.15.40:0/2291929669 reap_dead start 2024-03-08T06:06:36.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.178+0000 7f8debe00640 1 -- 172.21.15.40:0/2291929669 shutdown_connections 2024-03-08T06:06:36.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.178+0000 7f8debe00640 1 -- 172.21.15.40:0/2291929669 >> 172.21.15.40:0/2291929669 conn(0x7f8dec06ac50 msgr2=0x7f8dec0b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:36.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.178+0000 7f8debe00640 1 -- 172.21.15.40:0/2291929669 shutdown_connections 2024-03-08T06:06:36.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.178+0000 7f8debe00640 1 -- 172.21.15.40:0/2291929669 wait complete. 2024-03-08T06:06:36.195 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-08T06:06:36.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.421+0000 7f2944600640 1 Processor -- start 2024-03-08T06:06:36.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.422+0000 7f2944600640 1 -- start start 2024-03-08T06:06:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.423+0000 7f2944600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f293c145250 0x7f293c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.423+0000 7f2944600640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f293c0717d0 0x7f293c071c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.423+0000 7f2944600640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293c0cc690 con 0x7f293c0cd730 2024-03-08T06:06:36.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.423+0000 7f2944600640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293c072160 con 0x7f293c0717d0 2024-03-08T06:06:36.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.423+0000 7f2944600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293c0cc970 con 0x7f293c145250 2024-03-08T06:06:36.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.424+0000 7f2941200640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f293c0cd730 0x7f293c06eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36522/0 (socket says 172.21.15.40:36522) 2024-03-08T06:06:36.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.424+0000 7f2941200640 1 -- 172.21.15.40:0/3126504434 learned_addr learned my addr 172.21.15.40:0/3126504434 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:36.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.424+0000 7f2942600640 1 --2- 172.21.15.40:0/3126504434 >> v2:172.21.15.53:3300/0 conn(0x7f293c0717d0 0x7f293c071c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.424+0000 7f2941c00640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f293c145250 0x7f293c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.424+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278213123 0 0) 0x7f293c0cc690 con 0x7f293c0cd730 2024-03-08T06:06:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.424+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2924003520 con 0x7f293c0cd730 2024-03-08T06:06:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1337894071 0 0) 0x7f2924003520 con 0x7f293c0cd730 2024-03-08T06:06:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f293c145250 msgr2=0x7f293c0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f293c145250 0x7f293c0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 >> v2:172.21.15.53:3300/0 conn(0x7f293c0717d0 msgr2=0x7f293c071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 --2- 172.21.15.40:0/3126504434 >> v2:172.21.15.53:3300/0 conn(0x7f293c0717d0 0x7f293c071c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c0ccc50 con 0x7f293c0cd730 2024-03-08T06:06:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f2928001670 con 0x7f293c0cd730 2024-03-08T06:06:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 >> v1:172.21.15.40:6789/0 conn(0x7f293c0cd730 legacy=0x7f293c06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29240039b0 0x7f2924003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2924004800 0x7f2924004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f293c145250 0x7f29240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c072160 con 0x7f29240039b0 2024-03-08T06:06:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c0cc970 con 0x7f2924004800 2024-03-08T06:06:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292400b9c0 con 0x7f293c145250 2024-03-08T06:06:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.425+0000 7f2941c00640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29240039b0 0x7f2924003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.426+0000 7f2941200640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2924004800 0x7f2924004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.426+0000 7f2941c00640 1 -- 172.21.15.40:0/3126504434 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f293c145250 msgr2=0x7f29240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.426+0000 7f2941c00640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f293c145250 0x7f29240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.426+0000 7f2941c00640 1 -- 172.21.15.40:0/3126504434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2924004800 msgr2=0x7f2924004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.426+0000 7f2941c00640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2924004800 0x7f2924004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.426+0000 7f2941c00640 1 -- 172.21.15.40:0/3126504434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924003520 con 0x7f29240039b0 2024-03-08T06:06:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.426+0000 7f2941200640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2924004800 0x7f2924004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.427+0000 7f2941c00640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29240039b0 0x7f2924003dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2930009ac0 tx=0x7f2924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2290f2e62ba666 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.427+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2930039070 con 0x7f29240039b0 2024-03-08T06:06:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.427+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293002c6d0 con 0x7f29240039b0 2024-03-08T06:06:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.427+0000 7f293be00640 1 -- 172.21.15.40:0/3126504434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2930009690 con 0x7f29240039b0 2024-03-08T06:06:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.428+0000 7f2944600640 1 -- 172.21.15.40:0/3126504434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29240039b0 msgr2=0x7f2924003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.428+0000 7f2944600640 1 --2- 172.21.15.40:0/3126504434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29240039b0 0x7f2924003dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2930009ac0 tx=0x7f2924005830 comp rx=0 tx=0).stop 2024-03-08T06:06:36.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.428+0000 7f2942600640 1 -- 172.21.15.40:0/3126504434 reap_dead start 2024-03-08T06:06:36.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.428+0000 7f2944600640 1 -- 172.21.15.40:0/3126504434 shutdown_connections 2024-03-08T06:06:36.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.428+0000 7f2944600640 1 -- 172.21.15.40:0/3126504434 >> 172.21.15.40:0/3126504434 conn(0x7f293c06ae30 msgr2=0x7f293c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:36.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.428+0000 7f2944600640 1 -- 172.21.15.40:0/3126504434 shutdown_connections 2024-03-08T06:06:36.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.428+0000 7f2944600640 1 -- 172.21.15.40:0/3126504434 wait complete. 2024-03-08T06:06:36.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.430+0000 7f2944600640 1 Processor -- start 2024-03-08T06:06:36.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.430+0000 7f2944600640 1 -- start start 2024-03-08T06:06:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.431+0000 7f2944600640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29240039b0 0x7f293c13eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.431+0000 7f2944600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f293c0a8a20 0x7f293c0a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.431+0000 7f2944600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f293c15a640 0x7f293c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.431+0000 7f2944600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c145630 con 0x7f293c0a8a20 2024-03-08T06:06:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.431+0000 7f2944600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c0cd210 con 0x7f293c15a640 2024-03-08T06:06:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.431+0000 7f2944600640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293c1457b0 con 0x7f29240039b0 2024-03-08T06:06:36.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.431+0000 7f2941200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f293c0a8a20 0x7f293c0a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.431+0000 7f2941200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f293c0a8a20 0x7f293c0a8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36370/0 (socket says 172.21.15.40:36370) 2024-03-08T06:06:36.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.431+0000 7f2941200640 1 -- 172.21.15.40:0/278112308 learned_addr learned my addr 172.21.15.40:0/278112308 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:36.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.431+0000 7f2942600640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f293c15a640 0x7f293c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.431+0000 7f2941c00640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29240039b0 0x7f293c13eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2941200640 1 -- 172.21.15.40:0/278112308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29240039b0 msgr2=0x7f293c13eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2941200640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29240039b0 0x7f293c13eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2941200640 1 -- 172.21.15.40:0/278112308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f293c15a640 msgr2=0x7f293c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2941200640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f293c15a640 0x7f293c15aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2941200640 1 -- 172.21.15.40:0/278112308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c15b510 con 0x7f293c0a8a20 2024-03-08T06:06:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2941c00640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29240039b0 0x7f293c13eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2941200640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f293c0a8a20 0x7f293c0a8ea0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f292800d770 tx=0x7f292800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2939600640 1 -- 172.21.15.40:0/278112308 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928019070 con 0x7f293c0a8a20 2024-03-08T06:06:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2939600640 1 -- 172.21.15.40:0/278112308 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29280074e0 con 0x7f293c0a8a20 2024-03-08T06:06:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2939600640 1 -- 172.21.15.40:0/278112308 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928004a00 con 0x7f293c0a8a20 2024-03-08T06:06:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2944600640 1 -- 172.21.15.40:0/278112308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293c0a9600 con 0x7f293c0a8a20 2024-03-08T06:06:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.432+0000 7f2944600640 1 -- 172.21.15.40:0/278112308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293c0a9bc0 con 0x7f293c0a8a20 2024-03-08T06:06:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.434+0000 7f2944600640 1 -- 172.21.15.40:0/278112308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293c067cb0 con 0x7f293c0a8a20 2024-03-08T06:06:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.434+0000 7f2939600640 1 -- 172.21.15.40:0/278112308 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2928004ba0 con 0x7f293c0a8a20 2024-03-08T06:06:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.435+0000 7f2939600640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f291c06b4d0 0x7f291c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.435+0000 7f2939600640 1 -- 172.21.15.40:0/278112308 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2928089830 con 0x7f293c0a8a20 2024-03-08T06:06:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.436+0000 7f2941c00640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f291c06b4d0 0x7f291c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.436+0000 7f2941c00640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f291c06b4d0 0x7f291c06d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2930036000 tx=0x7f2930030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.440+0000 7f2939600640 1 -- 172.21.15.40:0/278112308 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f293c15b510 con 0x7f293c0a8a20 2024-03-08T06:06:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.639+0000 7f2944600640 1 -- 172.21.15.40:0/278112308 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f293c15b000 con 0x7f291c06b4d0 2024-03-08T06:06:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.640+0000 7f2939600640 1 -- 172.21.15.40:0/278112308 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2930004140 con 0x7f291c06b4d0 2024-03-08T06:06:36.641 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2024-03-08T06:06:36.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.643+0000 7f2944600640 1 -- 172.21.15.40:0/278112308 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f291c06b4d0 msgr2=0x7f291c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.643+0000 7f2944600640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f291c06b4d0 0x7f291c06d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2930036000 tx=0x7f2930030000 comp rx=0 tx=0).stop 2024-03-08T06:06:36.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.643+0000 7f2944600640 1 -- 172.21.15.40:0/278112308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f293c0a8a20 msgr2=0x7f293c0a8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.643+0000 7f2944600640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f293c0a8a20 0x7f293c0a8ea0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f292800d770 tx=0x7f292800dc30 comp rx=0 tx=0).stop 2024-03-08T06:06:36.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.643+0000 7f2944600640 1 -- 172.21.15.40:0/278112308 shutdown_connections 2024-03-08T06:06:36.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.643+0000 7f2944600640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f291c06b4d0 0x7f291c06d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.643+0000 7f2944600640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f293c15a640 0x7f293c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.643+0000 7f2944600640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f293c0a8a20 0x7f293c0a8ea0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.643+0000 7f2944600640 1 --2- 172.21.15.40:0/278112308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29240039b0 0x7f293c13eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.643+0000 7f2944600640 1 -- 172.21.15.40:0/278112308 >> 172.21.15.40:0/278112308 conn(0x7f293c06ae30 msgr2=0x7f293c142a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:36.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.644+0000 7f2944600640 1 -- 172.21.15.40:0/278112308 shutdown_connections 2024-03-08T06:06:36.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.644+0000 7f2944600640 1 -- 172.21.15.40:0/278112308 wait complete. 2024-03-08T06:06:36.662 INFO:tasks.ceph:Scrubbing osd.5 2024-03-08T06:06:36.662 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:06:36.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.892+0000 7f3a4d800640 1 Processor -- start 2024-03-08T06:06:36.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.892+0000 7f3a4d800640 1 -- start start 2024-03-08T06:06:36.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.894+0000 7f3a4d800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a4806e8a0 0x7f3a480b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.894+0000 7f3a4d800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a480b37e0 con 0x7f3a4815e5a0 2024-03-08T06:06:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.894+0000 7f3a4d800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4815ea30 con 0x7f3a4806ec80 2024-03-08T06:06:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.894+0000 7f3a4d800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480cf740 con 0x7f3a4806e8a0 2024-03-08T06:06:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.894+0000 7f3a47400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3a4815e5a0 0x7f3a480cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50134/0 (socket says 172.21.15.40:50134) 2024-03-08T06:06:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.894+0000 7f3a47400640 1 -- 172.21.15.40:0/3889862169 learned_addr learned my addr 172.21.15.40:0/3889862169 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a47e00640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a4806e8a0 0x7f3a480b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428955409 0 0) 0x7f3a480b37e0 con 0x7f3a4815e5a0 2024-03-08T06:06:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a2c003520 con 0x7f3a4815e5a0 2024-03-08T06:06:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2047887301 0 0) 0x7f3a2c003520 con 0x7f3a4815e5a0 2024-03-08T06:06:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a4806e8a0 msgr2=0x7f3a480b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a46a00640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a4806e8a0 0x7f3a480b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 >> v1:172.21.15.53:6789/0 conn(0x7f3a4806ec80 legacy=0x7f3a480cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4815ed10 con 0x7f3a4815e5a0 2024-03-08T06:06:36.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f3a34001690 con 0x7f3a4815e5a0 2024-03-08T06:06:36.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 >> v1:172.21.15.40:6789/0 conn(0x7f3a4815e5a0 legacy=0x7f3a480cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a46a00640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a4806e8a0 0x7f3a2c003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.895+0000 7f3a46a00640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a2c004020 0x7f3a2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a46a00640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a2c004d40 0x7f3a2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480cf740 con 0x7f3a2c004020 2024-03-08T06:06:36.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c00bc00 con 0x7f3a2c004d40 2024-03-08T06:06:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c00be00 con 0x7f3a4806e8a0 2024-03-08T06:06:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a47400640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a2c004020 0x7f3a2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a47e00640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a4806e8a0 0x7f3a2c003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a4ce00640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a2c004d40 0x7f3a2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a47400640 1 -- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a4806e8a0 msgr2=0x7f3a2c003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a47400640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a4806e8a0 0x7f3a2c003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a47400640 1 -- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a2c004d40 msgr2=0x7f3a2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:36.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a47400640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a2c004d40 0x7f3a2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a47400640 1 -- 172.21.15.40:0/3889862169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c003520 con 0x7f3a2c004020 2024-03-08T06:06:36.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a4ce00640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a2c004d40 0x7f3a2c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:36.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.896+0000 7f3a47e00640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a4806e8a0 0x7f3a2c003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.897+0000 7f3a47400640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a2c004020 0x7f3a2c004800 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3a480b39b0 tx=0x7f3a2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466c4669c355a5d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.898+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34007c80 con 0x7f3a2c004020 2024-03-08T06:06:36.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.898+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a34030070 con 0x7f3a2c004020 2024-03-08T06:06:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.898+0000 7f3a46a00640 1 -- 172.21.15.40:0/3889862169 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34005d10 con 0x7f3a2c004020 2024-03-08T06:06:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.898+0000 7f3a4d800640 1 -- 172.21.15.40:0/3889862169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a2c004020 msgr2=0x7f3a2c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.898+0000 7f3a4d800640 1 --2- 172.21.15.40:0/3889862169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a2c004020 0x7f3a2c004800 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3a480b39b0 tx=0x7f3a2c005830 comp rx=0 tx=0).stop 2024-03-08T06:06:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.898+0000 7f3a4ce00640 1 -- 172.21.15.40:0/3889862169 reap_dead start 2024-03-08T06:06:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.898+0000 7f3a4d800640 1 -- 172.21.15.40:0/3889862169 shutdown_connections 2024-03-08T06:06:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.898+0000 7f3a4d800640 1 -- 172.21.15.40:0/3889862169 >> 172.21.15.40:0/3889862169 conn(0x7f3a4806ac50 msgr2=0x7f3a4806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:36.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.899+0000 7f3a4d800640 1 -- 172.21.15.40:0/3889862169 shutdown_connections 2024-03-08T06:06:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.899+0000 7f3a4d800640 1 -- 172.21.15.40:0/3889862169 wait complete. 2024-03-08T06:06:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.900+0000 7f3a4d800640 1 Processor -- start 2024-03-08T06:06:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.900+0000 7f3a4d800640 1 -- start start 2024-03-08T06:06:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.901+0000 7f3a4d800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a2c004020 0x7f3a48144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.901+0000 7f3a4d800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a48149640 0x7f3a48145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.901+0000 7f3a4d800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a48142b40 0x7f3a48142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.901+0000 7f3a4d800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480cd540 con 0x7f3a2c004020 2024-03-08T06:06:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.901+0000 7f3a4d800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4815f2d0 con 0x7f3a48142b40 2024-03-08T06:06:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.901+0000 7f3a4d800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480cd6c0 con 0x7f3a48149640 2024-03-08T06:06:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.902+0000 7f3a47e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a2c004020 0x7f3a48144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.902+0000 7f3a47e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a2c004020 0x7f3a48144be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33862/0 (socket says 172.21.15.40:33862) 2024-03-08T06:06:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.902+0000 7f3a47e00640 1 -- 172.21.15.40:0/1028847622 learned_addr learned my addr 172.21.15.40:0/1028847622 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.902+0000 7f3a47400640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a48149640 0x7f3a48145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.902+0000 7f3a4ce00640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a48142b40 0x7f3a48142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.902+0000 7f3a47e00640 1 -- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a48149640 msgr2=0x7f3a48145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.902+0000 7f3a47e00640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a48149640 0x7f3a48145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.902+0000 7f3a47e00640 1 -- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a48142b40 msgr2=0x7f3a48142fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.902+0000 7f3a47e00640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a48142b40 0x7f3a48142fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.902+0000 7f3a47e00640 1 -- 172.21.15.40:0/1028847622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a48149ac0 con 0x7f3a2c004020 2024-03-08T06:06:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.902+0000 7f3a47400640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a48149640 0x7f3a48145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:36.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.903+0000 7f3a47e00640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a2c004020 0x7f3a48144be0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3a380108e0 tx=0x7f3a38010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.903+0000 7f3a3fe00640 1 -- 172.21.15.40:0/1028847622 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3800f040 con 0x7f3a2c004020 2024-03-08T06:06:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.903+0000 7f3a3fe00640 1 -- 172.21.15.40:0/1028847622 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3800ec20 con 0x7f3a2c004020 2024-03-08T06:06:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.903+0000 7f3a3fe00640 1 -- 172.21.15.40:0/1028847622 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3801c400 con 0x7f3a2c004020 2024-03-08T06:06:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.903+0000 7f3a4d800640 1 -- 172.21.15.40:0/1028847622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a48149d90 con 0x7f3a2c004020 2024-03-08T06:06:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.903+0000 7f3a4d800640 1 -- 172.21.15.40:0/1028847622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4814a350 con 0x7f3a2c004020 2024-03-08T06:06:36.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.904+0000 7f3a4d800640 1 -- 172.21.15.40:0/1028847622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a48075090 con 0x7f3a2c004020 2024-03-08T06:06:36.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.905+0000 7f3a3fe00640 1 -- 172.21.15.40:0/1028847622 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a3801c5a0 con 0x7f3a2c004020 2024-03-08T06:06:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.906+0000 7f3a3fe00640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3a2806b500 0x7f3a2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.906+0000 7f3a3fe00640 1 -- 172.21.15.40:0/1028847622 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3a3808a620 con 0x7f3a2c004020 2024-03-08T06:06:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.906+0000 7f3a3fe00640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] conn(0x7f3a280713c0 0x7f3a280737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.906+0000 7f3a3fe00640 1 -- 172.21.15.40:0/1028847622 --> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a28073e90 con 0x7f3a280713c0 2024-03-08T06:06:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.906+0000 7f3a47400640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3a2806b500 0x7f3a2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.906+0000 7f3a3fe00640 1 -- 172.21.15.40:0/1028847622 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a38091050 con 0x7f3a2c004020 2024-03-08T06:06:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.907+0000 7f3a4ce00640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] conn(0x7f3a280713c0 0x7f3a280737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.907+0000 7f3a47400640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3a2806b500 0x7f3a2806d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a34034000 tx=0x7f3a34009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.908+0000 7f3a4ce00640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] conn(0x7f3a280713c0 0x7f3a280737e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:36.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.911+0000 7f3a3fe00640 1 -- 172.21.15.40:0/1028847622 <== osd.5 v2:172.21.15.53:6800/3704060731 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3a28073e90 con 0x7f3a280713c0 2024-03-08T06:06:36.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.925+0000 7f3a4d800640 1 -- 172.21.15.40:0/1028847622 --> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3a480b3080 con 0x7f3a280713c0 2024-03-08T06:06:36.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.926+0000 7f3a3fe00640 1 -- 172.21.15.40:0/1028847622 <== osd.5 v2:172.21.15.53:6800/3704060731 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3a480b3080 con 0x7f3a280713c0 2024-03-08T06:06:36.927 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:06:36.927 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:06:36.927 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:06:36.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.927+0000 7f3a4d800640 1 -- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] conn(0x7f3a280713c0 msgr2=0x7f3a280737e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.927+0000 7f3a4d800640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.53:6800/3704060731,v1:172.21.15.53:6801/3704060731] conn(0x7f3a280713c0 0x7f3a280737e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.927+0000 7f3a4d800640 1 -- 172.21.15.40:0/1028847622 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3a2806b500 msgr2=0x7f3a2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.927+0000 7f3a4d800640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f3a2806b500 0x7f3a2806d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a34034000 tx=0x7f3a34009030 comp rx=0 tx=0).stop 2024-03-08T06:06:36.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.927+0000 7f3a4d800640 1 -- 172.21.15.40:0/1028847622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a2c004020 msgr2=0x7f3a48144be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:36.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.927+0000 7f3a4d800640 1 --2- 172.21.15.40:0/1028847622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a2c004020 0x7f3a48144be0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3a380108e0 tx=0x7f3a38010da0 comp rx=0 tx=0).stop 2024-03-08T06:06:36.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.927+0000 7f3a4ce00640 1 -- 172.21.15.40:0/1028847622 reap_dead start 2024-03-08T06:06:36.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.928+0000 7f3a4d800640 1 -- 172.21.15.40:0/1028847622 shutdown_connections 2024-03-08T06:06:36.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.928+0000 7f3a4d800640 1 -- 172.21.15.40:0/1028847622 >> 172.21.15.40:0/1028847622 conn(0x7f3a4806ac50 msgr2=0x7f3a4814c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:36.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.928+0000 7f3a4d800640 1 -- 172.21.15.40:0/1028847622 shutdown_connections 2024-03-08T06:06:36.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:36.928+0000 7f3a4d800640 1 -- 172.21.15.40:0/1028847622 wait complete. 2024-03-08T06:06:36.948 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-08T06:06:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.163+0000 7f14afe00640 1 Processor -- start 2024-03-08T06:06:37.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.164+0000 7f14afe00640 1 -- start start 2024-03-08T06:06:37.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.165+0000 7f14afe00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f14b01451b0 0x7f14b015ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.165+0000 7f14afe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14b0071b10 0x7f14b015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.165+0000 7f14afe00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b015cf70 con 0x7f14b006f050 2024-03-08T06:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.165+0000 7f14afe00640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b006d600 con 0x7f14b01451b0 2024-03-08T06:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.165+0000 7f14afe00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b006d8e0 con 0x7f14b0071b10 2024-03-08T06:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.166+0000 7f14aea00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f14b01451b0 0x7f14b015ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.166+0000 7f14aea00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f14b01451b0 0x7f14b015ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33876/0 (socket says 172.21.15.40:33876) 2024-03-08T06:06:37.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.166+0000 7f14aea00640 1 -- 172.21.15.40:0/2809077544 learned_addr learned my addr 172.21.15.40:0/2809077544 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:37.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.166+0000 7f14af400640 1 --2- 172.21.15.40:0/2809077544 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14b0071b10 0x7f14b015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.166+0000 7f14aea00640 1 -- 172.21.15.40:0/2809077544 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14b0071b10 msgr2=0x7f14b015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:37.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.166+0000 7f14aea00640 1 --2- 172.21.15.40:0/2809077544 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14b0071b10 0x7f14b015d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.166+0000 7f14aea00640 1 -- 172.21.15.40:0/2809077544 >> v1:172.21.15.53:6789/0 conn(0x7f14b006f050 legacy=0x7f14b0071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:37.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.166+0000 7f14aea00640 1 -- 172.21.15.40:0/2809077544 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b006dbc0 con 0x7f14b01451b0 2024-03-08T06:06:37.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.166+0000 7f14af400640 1 --2- 172.21.15.40:0/2809077544 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14b0071b10 0x7f14b015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.167+0000 7f14aea00640 1 --2- 172.21.15.40:0/2809077544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b01451b0 0x7f14b015ca30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009870 tx=0x7f14a002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f63f320384e4abf server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.168+0000 7f14ae000640 1 -- 172.21.15.40:0/2809077544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0035020 con 0x7f14b01451b0 2024-03-08T06:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.168+0000 7f14ae000640 1 -- 172.21.15.40:0/2809077544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a0031640 con 0x7f14b01451b0 2024-03-08T06:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.168+0000 7f14ae000640 1 -- 172.21.15.40:0/2809077544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a002d040 con 0x7f14b01451b0 2024-03-08T06:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.168+0000 7f14afe00640 1 -- 172.21.15.40:0/2809077544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b01451b0 msgr2=0x7f14b015ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.168+0000 7f14afe00640 1 --2- 172.21.15.40:0/2809077544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b01451b0 0x7f14b015ca30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009870 tx=0x7f14a002b8f0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.169+0000 7f14afe00640 1 -- 172.21.15.40:0/2809077544 shutdown_connections 2024-03-08T06:06:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.169+0000 7f14afe00640 1 --2- 172.21.15.40:0/2809077544 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14b0071b10 0x7f14b015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.169+0000 7f14afe00640 1 --2- 172.21.15.40:0/2809077544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b01451b0 0x7f14b015ca30 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.169+0000 7f14afe00640 1 -- 172.21.15.40:0/2809077544 >> 172.21.15.40:0/2809077544 conn(0x7f14b00b1590 msgr2=0x7f14b00b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.169+0000 7f14afe00640 1 -- 172.21.15.40:0/2809077544 shutdown_connections 2024-03-08T06:06:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.169+0000 7f14afe00640 1 -- 172.21.15.40:0/2809077544 wait complete. 2024-03-08T06:06:37.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.170+0000 7f14afe00640 1 Processor -- start 2024-03-08T06:06:37.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.171+0000 7f14afe00640 1 -- start start 2024-03-08T06:06:37.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.171+0000 7f14afe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b006f050 0x7f14b0143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.171+0000 7f14afe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b0071b10 0x7f14b015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.171+0000 7f14afe00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14b01451b0 0x7f14b01e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14afe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b0145710 con 0x7f14b0071b10 2024-03-08T06:06:37.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14afe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b0145590 con 0x7f14b006f050 2024-03-08T06:06:37.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14afe00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b015d6e0 con 0x7f14b01451b0 2024-03-08T06:06:37.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14a7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b0071b10 0x7f14b015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14a7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b0071b10 0x7f14b015b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33890/0 (socket says 172.21.15.40:33890) 2024-03-08T06:06:37.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14a7e00640 1 -- 172.21.15.40:0/2419093340 learned_addr learned my addr 172.21.15.40:0/2419093340 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:37.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14aea00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b006f050 0x7f14b0143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14af400640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14b01451b0 0x7f14b01e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14a7e00640 1 -- 172.21.15.40:0/2419093340 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14b01451b0 msgr2=0x7f14b01e5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14a7e00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14b01451b0 0x7f14b01e5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14a7e00640 1 -- 172.21.15.40:0/2419093340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b006f050 msgr2=0x7f14b0143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14a7e00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b006f050 0x7f14b0143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14a7e00640 1 -- 172.21.15.40:0/2419093340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b0142a10 con 0x7f14b0071b10 2024-03-08T06:06:37.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.172+0000 7f14aea00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b006f050 0x7f14b0143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:37.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.173+0000 7f14a7e00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b0071b10 0x7f14b015b2b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f149800c400 tx=0x7f149800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:37.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.173+0000 7f14a6a00640 1 -- 172.21.15.40:0/2419093340 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498011790 con 0x7f14b0071b10 2024-03-08T06:06:37.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.173+0000 7f14a6a00640 1 -- 172.21.15.40:0/2419093340 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1498017070 con 0x7f14b0071b10 2024-03-08T06:06:37.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.173+0000 7f14a6a00640 1 -- 172.21.15.40:0/2419093340 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498011bb0 con 0x7f14b0071b10 2024-03-08T06:06:37.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.173+0000 7f14afe00640 1 -- 172.21.15.40:0/2419093340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b0142cf0 con 0x7f14b0071b10 2024-03-08T06:06:37.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.173+0000 7f14afe00640 1 -- 172.21.15.40:0/2419093340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b01432b0 con 0x7f14b0071b10 2024-03-08T06:06:37.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.174+0000 7f14afe00640 1 -- 172.21.15.40:0/2419093340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b0075090 con 0x7f14b0071b10 2024-03-08T06:06:37.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.175+0000 7f14a6a00640 1 -- 172.21.15.40:0/2419093340 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f149801c460 con 0x7f14b0071b10 2024-03-08T06:06:37.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.176+0000 7f14a6a00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f148c06b4d0 0x7f148c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.176+0000 7f14a6a00640 1 -- 172.21.15.40:0/2419093340 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1498089e50 con 0x7f14b0071b10 2024-03-08T06:06:37.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.176+0000 7f14aea00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f148c06b4d0 0x7f148c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.177+0000 7f14aea00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f148c06b4d0 0x7f148c06d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14a0036000 tx=0x7f14a003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:37.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.182+0000 7f14a6a00640 1 -- 172.21.15.40:0/2419093340 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1498058f30 con 0x7f14b0071b10 2024-03-08T06:06:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.380+0000 7f14afe00640 1 -- 172.21.15.40:0/2419093340 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f14b01427c0 con 0x7f148c06b4d0 2024-03-08T06:06:37.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.380+0000 7f14a6a00640 1 -- 172.21.15.40:0/2419093340 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f14b01427c0 con 0x7f148c06b4d0 2024-03-08T06:06:37.381 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2024-03-08T06:06:37.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 -- 172.21.15.40:0/2419093340 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f148c06b4d0 msgr2=0x7f148c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f148c06b4d0 0x7f148c06d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14a0036000 tx=0x7f14a003c040 comp rx=0 tx=0).stop 2024-03-08T06:06:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 -- 172.21.15.40:0/2419093340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b0071b10 msgr2=0x7f14b015b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b0071b10 0x7f14b015b2b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f149800c400 tx=0x7f149800c8c0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 -- 172.21.15.40:0/2419093340 shutdown_connections 2024-03-08T06:06:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f148c06b4d0 0x7f148c06d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14b01451b0 0x7f14b01e5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14b0071b10 0x7f14b015b2b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 --2- 172.21.15.40:0/2419093340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b006f050 0x7f14b0143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 -- 172.21.15.40:0/2419093340 >> 172.21.15.40:0/2419093340 conn(0x7f14b00b1590 msgr2=0x7f14b015c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 -- 172.21.15.40:0/2419093340 shutdown_connections 2024-03-08T06:06:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.383+0000 7f14afe00640 1 -- 172.21.15.40:0/2419093340 wait complete. 2024-03-08T06:06:37.403 INFO:tasks.ceph:Scrubbing osd.6 2024-03-08T06:06:37.404 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:06:37.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.629+0000 7f605e200640 1 Processor -- start 2024-03-08T06:06:37.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.629+0000 7f605e200640 1 -- start start 2024-03-08T06:06:37.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.631+0000 7f605e200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6058072760 0x7f605806d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.631+0000 7f605e200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6058071e40 0x7f60580b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.631+0000 7f605e200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60580ae650 con 0x7f6058071a60 2024-03-08T06:06:37.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.631+0000 7f605e200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60580aee60 con 0x7f6058071e40 2024-03-08T06:06:37.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.631+0000 7f605e200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60580ad460 con 0x7f6058072760 2024-03-08T06:06:37.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.632+0000 7f6056a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6058072760 0x7f605806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.632+0000 7f6057e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6058071e40 0x7f60580b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.632+0000 7f6056a00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6058072760 0x7f605806d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:60082/0 (socket says 172.21.15.40:60082) 2024-03-08T06:06:37.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.632+0000 7f6056a00640 1 -- 172.21.15.40:0/1890069987 learned_addr learned my addr 172.21.15.40:0/1890069987 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:37.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.632+0000 7f6056a00640 1 -- 172.21.15.40:0/1890069987 >> v2:172.21.15.53:3300/0 conn(0x7f6058071e40 msgr2=0x7f60580b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:37.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.632+0000 7f6056a00640 1 --2- 172.21.15.40:0/1890069987 >> v2:172.21.15.53:3300/0 conn(0x7f6058071e40 0x7f60580b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.632+0000 7f6056a00640 1 -- 172.21.15.40:0/1890069987 >> v1:172.21.15.53:6789/0 conn(0x7f6058071a60 legacy=0x7f60580adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.632+0000 7f6056a00640 1 -- 172.21.15.40:0/1890069987 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60580ad740 con 0x7f6058072760 2024-03-08T06:06:37.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.634+0000 7f6056a00640 1 --2- 172.21.15.40:0/1890069987 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6058072760 0x7f605806d850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6044009870 tx=0x7f604402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f606d2741bae0007 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:37.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.634+0000 7f6056000640 1 -- 172.21.15.40:0/1890069987 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6044035020 con 0x7f6058072760 2024-03-08T06:06:37.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.634+0000 7f6056000640 1 -- 172.21.15.40:0/1890069987 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6044031640 con 0x7f6058072760 2024-03-08T06:06:37.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.634+0000 7f6056000640 1 -- 172.21.15.40:0/1890069987 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604402d040 con 0x7f6058072760 2024-03-08T06:06:37.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.635+0000 7f605e200640 1 -- 172.21.15.40:0/1890069987 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6058072760 msgr2=0x7f605806d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.635+0000 7f605e200640 1 --2- 172.21.15.40:0/1890069987 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6058072760 0x7f605806d850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6044009870 tx=0x7f604402b8f0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.635+0000 7f605e200640 1 -- 172.21.15.40:0/1890069987 shutdown_connections 2024-03-08T06:06:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.635+0000 7f605e200640 1 --2- 172.21.15.40:0/1890069987 >> v2:172.21.15.53:3300/0 conn(0x7f6058071e40 0x7f60580b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.635+0000 7f605e200640 1 --2- 172.21.15.40:0/1890069987 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6058072760 0x7f605806d850 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.635+0000 7f605e200640 1 -- 172.21.15.40:0/1890069987 >> 172.21.15.40:0/1890069987 conn(0x7f60580cfaa0 msgr2=0x7f60580cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.635+0000 7f605e200640 1 -- 172.21.15.40:0/1890069987 shutdown_connections 2024-03-08T06:06:37.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.635+0000 7f605e200640 1 -- 172.21.15.40:0/1890069987 wait complete. 2024-03-08T06:06:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.637+0000 7f605e200640 1 Processor -- start 2024-03-08T06:06:37.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.637+0000 7f605e200640 1 -- start start 2024-03-08T06:06:37.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.638+0000 7f605e200640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6058071a60 0x7f60581476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.638+0000 7f605e200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6058071e40 0x7f60581603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.638+0000 7f605e200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6058072760 0x7f6058160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.638+0000 7f605e200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60580ad110 con 0x7f6058072760 2024-03-08T06:06:37.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.638+0000 7f605e200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60580aeb10 con 0x7f6058071e40 2024-03-08T06:06:37.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.638+0000 7f605e200640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60580b3090 con 0x7f6058071a60 2024-03-08T06:06:37.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.639+0000 7f6057e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6058072760 0x7f6058160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.639+0000 7f6057e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6058072760 0x7f6058160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33892/0 (socket says 172.21.15.40:33892) 2024-03-08T06:06:37.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.639+0000 7f6057e00640 1 -- 172.21.15.40:0/2023453680 learned_addr learned my addr 172.21.15.40:0/2023453680 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:37.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.639+0000 7f6056a00640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6058071e40 0x7f60581603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.639+0000 7f6057400640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6058071a60 0x7f60581476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.639+0000 7f6057e00640 1 -- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6058071a60 msgr2=0x7f60581476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.639+0000 7f6057e00640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6058071a60 0x7f60581476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.639+0000 7f6057e00640 1 -- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6058071e40 msgr2=0x7f60581603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.639+0000 7f6057e00640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6058071e40 0x7f60581603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.639+0000 7f6057e00640 1 -- 172.21.15.40:0/2023453680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6058147bf0 con 0x7f6058072760 2024-03-08T06:06:37.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.639+0000 7f6056a00640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6058071e40 0x7f60581603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:37.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.640+0000 7f6057e00640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6058072760 0x7f6058160900 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f605000efc0 tx=0x7f605000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:37.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.640+0000 7f604f400640 1 -- 172.21.15.40:0/2023453680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050018020 con 0x7f6058072760 2024-03-08T06:06:37.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.640+0000 7f604f400640 1 -- 172.21.15.40:0/2023453680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6050014480 con 0x7f6058072760 2024-03-08T06:06:37.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.640+0000 7f604f400640 1 -- 172.21.15.40:0/2023453680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050014870 con 0x7f6058072760 2024-03-08T06:06:37.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.640+0000 7f605e200640 1 -- 172.21.15.40:0/2023453680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6058155f30 con 0x7f6058072760 2024-03-08T06:06:37.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.640+0000 7f605e200640 1 -- 172.21.15.40:0/2023453680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60581564f0 con 0x7f6058072760 2024-03-08T06:06:37.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.641+0000 7f605e200640 1 -- 172.21.15.40:0/2023453680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6058067cb0 con 0x7f6058072760 2024-03-08T06:06:37.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.642+0000 7f604f400640 1 -- 172.21.15.40:0/2023453680 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6050005d10 con 0x7f6058072760 2024-03-08T06:06:37.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.643+0000 7f604f400640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f603806b500 0x7f603806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.643+0000 7f604f400640 1 -- 172.21.15.40:0/2023453680 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f605008acf0 con 0x7f6058072760 2024-03-08T06:06:37.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.644+0000 7f6057400640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f603806b500 0x7f603806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.644+0000 7f604f400640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] conn(0x7f60380713c0 0x7f60380737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.644+0000 7f604f400640 1 -- 172.21.15.40:0/2023453680 --> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6050011e50 con 0x7f60380713c0 2024-03-08T06:06:37.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.644+0000 7f604f400640 1 -- 172.21.15.40:0/2023453680 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f605008d390 con 0x7f6058072760 2024-03-08T06:06:37.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.644+0000 7f6057400640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f603806b500 0x7f603806d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6048001620 tx=0x7f6048009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:37.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.644+0000 7f6056a00640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] conn(0x7f60380713c0 0x7f60380737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.645+0000 7f6056a00640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] conn(0x7f60380713c0 0x7f60380737e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:37.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.648+0000 7f604f400640 1 -- 172.21.15.40:0/2023453680 <== osd.6 v2:172.21.15.53:6824/2620899738 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6050011e50 con 0x7f60380713c0 2024-03-08T06:06:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.662+0000 7f605e200640 1 -- 172.21.15.40:0/2023453680 --> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f60580724d0 con 0x7f60380713c0 2024-03-08T06:06:37.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.663+0000 7f604f400640 1 -- 172.21.15.40:0/2023453680 <== osd.6 v2:172.21.15.53:6824/2620899738 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f60580724d0 con 0x7f60380713c0 2024-03-08T06:06:37.664 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:06:37.664 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:06:37.664 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:06:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.664+0000 7f605e200640 1 -- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] conn(0x7f60380713c0 msgr2=0x7f60380737e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.664+0000 7f605e200640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.53:6824/2620899738,v1:172.21.15.53:6825/2620899738] conn(0x7f60380713c0 0x7f60380737e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.664+0000 7f605e200640 1 -- 172.21.15.40:0/2023453680 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f603806b500 msgr2=0x7f603806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.664+0000 7f605e200640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f603806b500 0x7f603806d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6048001620 tx=0x7f6048009040 comp rx=0 tx=0).stop 2024-03-08T06:06:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.664+0000 7f605e200640 1 -- 172.21.15.40:0/2023453680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6058072760 msgr2=0x7f6058160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.664+0000 7f605e200640 1 --2- 172.21.15.40:0/2023453680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6058072760 0x7f6058160900 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f605000efc0 tx=0x7f605000c490 comp rx=0 tx=0).stop 2024-03-08T06:06:37.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.664+0000 7f6057e00640 1 -- 172.21.15.40:0/2023453680 reap_dead start 2024-03-08T06:06:37.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.664+0000 7f605e200640 1 -- 172.21.15.40:0/2023453680 shutdown_connections 2024-03-08T06:06:37.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.664+0000 7f605e200640 1 -- 172.21.15.40:0/2023453680 >> 172.21.15.40:0/2023453680 conn(0x7f60580cfaa0 msgr2=0x7f60580af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:37.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.664+0000 7f605e200640 1 -- 172.21.15.40:0/2023453680 shutdown_connections 2024-03-08T06:06:37.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.664+0000 7f605e200640 1 -- 172.21.15.40:0/2023453680 wait complete. 2024-03-08T06:06:37.683 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-08T06:06:37.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.905+0000 7f453ee00640 1 Processor -- start 2024-03-08T06:06:37.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.905+0000 7f453ee00640 1 -- start start 2024-03-08T06:06:37.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.907+0000 7f453ee00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f453806ec80 0x7f453815ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.907+0000 7f453ee00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f453815f210 0x7f453815b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.907+0000 7f453ee00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45380b49e0 con 0x7f453815a9a0 2024-03-08T06:06:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.907+0000 7f453ee00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453815e730 con 0x7f453815f210 2024-03-08T06:06:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.907+0000 7f453ee00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453815ea10 con 0x7f453806ec80 2024-03-08T06:06:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.907+0000 7f453da00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f453815a9a0 0x7f453806e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50138/0 (socket says 172.21.15.40:50138) 2024-03-08T06:06:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.907+0000 7f453da00640 1 -- 172.21.15.40:0/3375478530 learned_addr learned my addr 172.21.15.40:0/3375478530 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.907+0000 7f453e400640 1 --2- 172.21.15.40:0/3375478530 >> v2:172.21.15.53:3300/0 conn(0x7f453815f210 0x7f453815b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.907+0000 7f453d000640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f453806ec80 0x7f453815ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423693883 0 0) 0x7f45380b49e0 con 0x7f453815a9a0 2024-03-08T06:06:37.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4520003520 con 0x7f453815a9a0 2024-03-08T06:06:37.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2076653894 0 0) 0x7f4520003520 con 0x7f453815a9a0 2024-03-08T06:06:37.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f453806ec80 msgr2=0x7f453815ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:37.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f453806ec80 0x7f453815ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 >> v2:172.21.15.53:3300/0 conn(0x7f453815f210 msgr2=0x7f453815b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:37.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 --2- 172.21.15.40:0/3375478530 >> v2:172.21.15.53:3300/0 conn(0x7f453815f210 0x7f453815b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453815ecf0 con 0x7f453815a9a0 2024-03-08T06:06:37.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f452c001670 con 0x7f453815a9a0 2024-03-08T06:06:37.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 >> v1:172.21.15.40:6789/0 conn(0x7f453815a9a0 legacy=0x7f453806e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4520003a90 0x7f4520003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4520004800 0x7f4520004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 0x7f452000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453815e730 con 0x7f45200051c0 2024-03-08T06:06:37.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453815ea10 con 0x7f4520004800 2024-03-08T06:06:37.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452000e1c0 con 0x7f4520003a90 2024-03-08T06:06:37.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.908+0000 7f453d000640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f453806ec80 0x7f453815ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:06:37.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.909+0000 7f453e400640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 0x7f452000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.909+0000 7f453da00640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4520003a90 0x7f4520003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.909+0000 7f453e400640 1 -- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4520003a90 msgr2=0x7f4520003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:37.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.909+0000 7f453e400640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4520003a90 0x7f4520003eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.909+0000 7f453e400640 1 -- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4520004800 msgr2=0x7f4520004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:37.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.909+0000 7f453d000640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4520004800 0x7f4520004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.909+0000 7f453e400640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4520004800 0x7f4520004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.909+0000 7f453e400640 1 -- 172.21.15.40:0/3375478530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4520003520 con 0x7f45200051c0 2024-03-08T06:06:37.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.910+0000 7f453e400640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 0x7f452000dbd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4530009f90 tx=0x7f4520005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6e64444b3a2238 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:37.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.911+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530039070 con 0x7f45200051c0 2024-03-08T06:06:37.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.911+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453002cc20 con 0x7f45200051c0 2024-03-08T06:06:37.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.911+0000 7f4537e00640 1 -- 172.21.15.40:0/3375478530 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530009c10 con 0x7f45200051c0 2024-03-08T06:06:37.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.911+0000 7f453ee00640 1 -- 172.21.15.40:0/3375478530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 msgr2=0x7f452000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.911+0000 7f453ee00640 1 --2- 172.21.15.40:0/3375478530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 0x7f452000dbd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4530009f90 tx=0x7f4520005830 comp rx=0 tx=0).stop 2024-03-08T06:06:37.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.911+0000 7f453e400640 1 -- 172.21.15.40:0/3375478530 reap_dead start 2024-03-08T06:06:37.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.911+0000 7f453ee00640 1 -- 172.21.15.40:0/3375478530 shutdown_connections 2024-03-08T06:06:37.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.911+0000 7f453ee00640 1 -- 172.21.15.40:0/3375478530 >> 172.21.15.40:0/3375478530 conn(0x7f45380b1590 msgr2=0x7f45380b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:37.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.911+0000 7f453ee00640 1 -- 172.21.15.40:0/3375478530 shutdown_connections 2024-03-08T06:06:37.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.912+0000 7f453ee00640 1 -- 172.21.15.40:0/3375478530 wait complete. 2024-03-08T06:06:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.913+0000 7f453ee00640 1 Processor -- start 2024-03-08T06:06:37.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.913+0000 7f453ee00640 1 -- start start 2024-03-08T06:06:37.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.914+0000 7f453ee00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 0x7f453814ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.914+0000 7f453ee00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4538154a30 0x7f4538154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.914+0000 7f453ee00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f45380b2d40 0x7f45380b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.914+0000 7f453ee00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45380b45e0 con 0x7f45200051c0 2024-03-08T06:06:37.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.914+0000 7f453ee00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45380b4460 con 0x7f4538154a30 2024-03-08T06:06:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.914+0000 7f453ee00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45380b4760 con 0x7f45380b2d40 2024-03-08T06:06:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.915+0000 7f453da00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 0x7f453814ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.915+0000 7f453da00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 0x7f453814ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33904/0 (socket says 172.21.15.40:33904) 2024-03-08T06:06:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.915+0000 7f453da00640 1 -- 172.21.15.40:0/1909085696 learned_addr learned my addr 172.21.15.40:0/1909085696 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.915+0000 7f453da00640 1 -- 172.21.15.40:0/1909085696 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f45380b2d40 msgr2=0x7f45380b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.915+0000 7f453da00640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f45380b2d40 0x7f45380b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.915+0000 7f453da00640 1 -- 172.21.15.40:0/1909085696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4538154a30 msgr2=0x7f4538154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:37.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.915+0000 7f453da00640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4538154a30 0x7f4538154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:37.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.915+0000 7f453da00640 1 -- 172.21.15.40:0/1909085696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45380ae430 con 0x7f45200051c0 2024-03-08T06:06:37.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.915+0000 7f453da00640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 0x7f453814ee70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f452c00d750 tx=0x7f452c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:37.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.916+0000 7f4535600640 1 -- 172.21.15.40:0/1909085696 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c019070 con 0x7f45200051c0 2024-03-08T06:06:37.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.916+0000 7f4535600640 1 -- 172.21.15.40:0/1909085696 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c0074e0 con 0x7f45200051c0 2024-03-08T06:06:37.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.916+0000 7f4535600640 1 -- 172.21.15.40:0/1909085696 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c00de60 con 0x7f45200051c0 2024-03-08T06:06:37.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.916+0000 7f453ee00640 1 -- 172.21.15.40:0/1909085696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453815af20 con 0x7f45200051c0 2024-03-08T06:06:37.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.916+0000 7f453ee00640 1 -- 172.21.15.40:0/1909085696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453815b4a0 con 0x7f45200051c0 2024-03-08T06:06:37.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.917+0000 7f453ee00640 1 -- 172.21.15.40:0/1909085696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4538075090 con 0x7f45200051c0 2024-03-08T06:06:37.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.918+0000 7f4535600640 1 -- 172.21.15.40:0/1909085696 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f452c0049e0 con 0x7f45200051c0 2024-03-08T06:06:37.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.920+0000 7f4535600640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f451c06b500 0x7f451c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:37.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.920+0000 7f4535600640 1 -- 172.21.15.40:0/1909085696 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f452c089890 con 0x7f45200051c0 2024-03-08T06:06:37.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.920+0000 7f453d000640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f451c06b500 0x7f451c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:37.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.921+0000 7f453d000640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f451c06b500 0x7f451c06d9c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f45381529f0 tx=0x7f4528008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:37.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:37.924+0000 7f4535600640 1 -- 172.21.15.40:0/1909085696 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f452c058970 con 0x7f45200051c0 2024-03-08T06:06:38.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.124+0000 7f453ee00640 1 -- 172.21.15.40:0/1909085696 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f4538067920 con 0x7f451c06b500 2024-03-08T06:06:38.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.125+0000 7f4535600640 1 -- 172.21.15.40:0/1909085696 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4538067920 con 0x7f451c06b500 2024-03-08T06:06:38.126 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2024-03-08T06:06:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 -- 172.21.15.40:0/1909085696 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f451c06b500 msgr2=0x7f451c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f451c06b500 0x7f451c06d9c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f45381529f0 tx=0x7f4528008040 comp rx=0 tx=0).stop 2024-03-08T06:06:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 -- 172.21.15.40:0/1909085696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 msgr2=0x7f453814ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 0x7f453814ee70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f452c00d750 tx=0x7f452c00dc10 comp rx=0 tx=0).stop 2024-03-08T06:06:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 -- 172.21.15.40:0/1909085696 shutdown_connections 2024-03-08T06:06:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f451c06b500 0x7f451c06d9c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f45380b2d40 0x7f45380b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4538154a30 0x7f4538154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 --2- 172.21.15.40:0/1909085696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45200051c0 0x7f453814ee70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 -- 172.21.15.40:0/1909085696 >> 172.21.15.40:0/1909085696 conn(0x7f45380b1590 msgr2=0x7f45380cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:38.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 -- 172.21.15.40:0/1909085696 shutdown_connections 2024-03-08T06:06:38.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.128+0000 7f453ee00640 1 -- 172.21.15.40:0/1909085696 wait complete. 2024-03-08T06:06:38.147 INFO:tasks.ceph:Scrubbing osd.7 2024-03-08T06:06:38.147 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-08T06:06:38.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.371+0000 7f2506400640 1 Processor -- start 2024-03-08T06:06:38.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.372+0000 7f2506400640 1 -- start start 2024-03-08T06:06:38.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.373+0000 7f2506400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f25001584a0 0x7f2500071950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.374+0000 7f2506400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2500071e90 0x7f250015e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.374+0000 7f2506400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f250015f390 con 0x7f250015eb50 2024-03-08T06:06:38.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.374+0000 7f2506400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250006d820 con 0x7f25001584a0 2024-03-08T06:06:38.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.374+0000 7f2506400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250006db00 con 0x7f2500071e90 2024-03-08T06:06:38.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.374+0000 7f24ff400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f25001584a0 0x7f2500071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.374+0000 7f24fea00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2500071e90 0x7f250015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.374+0000 7f24fea00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2500071e90 0x7f250015e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:60124/0 (socket says 172.21.15.40:60124) 2024-03-08T06:06:38.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.374+0000 7f24fea00640 1 -- 172.21.15.40:0/1188799095 learned_addr learned my addr 172.21.15.40:0/1188799095 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:38.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.375+0000 7f24fea00640 1 -- 172.21.15.40:0/1188799095 >> v2:172.21.15.53:3300/0 conn(0x7f25001584a0 msgr2=0x7f2500071950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:38.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.375+0000 7f24fea00640 1 --2- 172.21.15.40:0/1188799095 >> v2:172.21.15.53:3300/0 conn(0x7f25001584a0 0x7f2500071950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.375+0000 7f24fea00640 1 -- 172.21.15.40:0/1188799095 >> v1:172.21.15.53:6789/0 conn(0x7f250015eb50 legacy=0x7f2500161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.375+0000 7f24fea00640 1 -- 172.21.15.40:0/1188799095 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250006dde0 con 0x7f2500071e90 2024-03-08T06:06:38.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.376+0000 7f24ff400640 1 --2- 172.21.15.40:0/1188799095 >> v2:172.21.15.53:3300/0 conn(0x7f25001584a0 0x7f2500071950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T06:06:38.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.377+0000 7f24fea00640 1 --2- 172.21.15.40:0/1188799095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2500071e90 0x7f250015e610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009870 tx=0x7f24e802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ce3eb155b2f4a2f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:38.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.377+0000 7f24fe000640 1 -- 172.21.15.40:0/1188799095 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8035020 con 0x7f2500071e90 2024-03-08T06:06:38.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.378+0000 7f24fe000640 1 -- 172.21.15.40:0/1188799095 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e8031640 con 0x7f2500071e90 2024-03-08T06:06:38.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.378+0000 7f24fe000640 1 -- 172.21.15.40:0/1188799095 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e802d040 con 0x7f2500071e90 2024-03-08T06:06:38.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.378+0000 7f2506400640 1 -- 172.21.15.40:0/1188799095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2500071e90 msgr2=0x7f250015e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.378+0000 7f2506400640 1 --2- 172.21.15.40:0/1188799095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2500071e90 0x7f250015e610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009870 tx=0x7f24e802b8f0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.378+0000 7f2506400640 1 -- 172.21.15.40:0/1188799095 shutdown_connections 2024-03-08T06:06:38.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.378+0000 7f2506400640 1 --2- 172.21.15.40:0/1188799095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2500071e90 0x7f250015e610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.378+0000 7f2506400640 1 --2- 172.21.15.40:0/1188799095 >> v2:172.21.15.53:3300/0 conn(0x7f25001584a0 0x7f2500071950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.378+0000 7f2506400640 1 -- 172.21.15.40:0/1188799095 >> 172.21.15.40:0/1188799095 conn(0x7f250006ae30 msgr2=0x7f250006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:38.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.378+0000 7f2506400640 1 -- 172.21.15.40:0/1188799095 shutdown_connections 2024-03-08T06:06:38.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.379+0000 7f2506400640 1 -- 172.21.15.40:0/1188799095 wait complete. 2024-03-08T06:06:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.380+0000 7f2506400640 1 Processor -- start 2024-03-08T06:06:38.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.380+0000 7f2506400640 1 -- start start 2024-03-08T06:06:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.381+0000 7f2506400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2500071e90 0x7f2500159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.381+0000 7f2506400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f25001584a0 0x7f25001f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.381+0000 7f2506400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f250015eb50 0x7f25001427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.381+0000 7f2506400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250006e5e0 con 0x7f2500071e90 2024-03-08T06:06:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.381+0000 7f2506400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2500158880 con 0x7f250015eb50 2024-03-08T06:06:38.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.381+0000 7f2506400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25001f4940 con 0x7f25001584a0 2024-03-08T06:06:38.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.382+0000 7f24ff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2500071e90 0x7f2500159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.382+0000 7f24ff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2500071e90 0x7f2500159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33914/0 (socket says 172.21.15.40:33914) 2024-03-08T06:06:38.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.382+0000 7f24ff400640 1 -- 172.21.15.40:0/3487866500 learned_addr learned my addr 172.21.15.40:0/3487866500 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:38.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.382+0000 7f24fea00640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f25001584a0 0x7f25001f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.382+0000 7f24ffe00640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f250015eb50 0x7f25001427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.382+0000 7f24ff400640 1 -- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f25001584a0 msgr2=0x7f25001f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.382+0000 7f24ff400640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f25001584a0 0x7f25001f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.382+0000 7f24ff400640 1 -- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f250015eb50 msgr2=0x7f25001427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.382+0000 7f24ff400640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f250015eb50 0x7f25001427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.382+0000 7f24ff400640 1 -- 172.21.15.40:0/3487866500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2500142d00 con 0x7f2500071e90 2024-03-08T06:06:38.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.382+0000 7f24fea00640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f25001584a0 0x7f25001f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:38.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.383+0000 7f24ff400640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2500071e90 0x7f2500159480 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f24f000efc0 tx=0x7f24f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:38.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.383+0000 7f24f7400640 1 -- 172.21.15.40:0/3487866500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0018020 con 0x7f2500071e90 2024-03-08T06:06:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.383+0000 7f24f7400640 1 -- 172.21.15.40:0/3487866500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f0014680 con 0x7f2500071e90 2024-03-08T06:06:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.383+0000 7f24f7400640 1 -- 172.21.15.40:0/3487866500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0014a50 con 0x7f2500071e90 2024-03-08T06:06:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.383+0000 7f2506400640 1 -- 172.21.15.40:0/3487866500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25001f4b70 con 0x7f2500071e90 2024-03-08T06:06:38.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.383+0000 7f2506400640 1 -- 172.21.15.40:0/3487866500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25001f5130 con 0x7f2500071e90 2024-03-08T06:06:38.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.384+0000 7f2506400640 1 -- 172.21.15.40:0/3487866500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25001f5370 con 0x7f2500071e90 2024-03-08T06:06:38.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.385+0000 7f24f7400640 1 -- 172.21.15.40:0/3487866500 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24f0005d10 con 0x7f2500071e90 2024-03-08T06:06:38.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.386+0000 7f24f7400640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f24dc06b4d0 0x7f24dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.386+0000 7f24f7400640 1 -- 172.21.15.40:0/3487866500 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f24f008ae00 con 0x7f2500071e90 2024-03-08T06:06:38.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.386+0000 7f24fea00640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f24dc06b4d0 0x7f24dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.386+0000 7f24f7400640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] conn(0x7f24dc071390 0x7f24dc0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.386+0000 7f24f7400640 1 -- 172.21.15.40:0/3487866500 --> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f24f0009a50 con 0x7f24dc071390 2024-03-08T06:06:38.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.386+0000 7f24f7400640 1 -- 172.21.15.40:0/3487866500 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24f008d420 con 0x7f2500071e90 2024-03-08T06:06:38.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.387+0000 7f24ffe00640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] conn(0x7f24dc071390 0x7f24dc0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.387+0000 7f24fea00640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f24dc06b4d0 0x7f24dc06d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f24e8036000 tx=0x7f24e8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:38.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.388+0000 7f24ffe00640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] conn(0x7f24dc071390 0x7f24dc0737b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:38.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.390+0000 7f24f7400640 1 -- 172.21.15.40:0/3487866500 <== osd.7 v2:172.21.15.53:6808/597382546 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f24f0009a50 con 0x7f24dc071390 2024-03-08T06:06:38.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.405+0000 7f2506400640 1 -- 172.21.15.40:0/3487866500 --> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f25001f5870 con 0x7f24dc071390 2024-03-08T06:06:38.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.406+0000 7f24f7400640 1 -- 172.21.15.40:0/3487866500 <== osd.7 v2:172.21.15.53:6808/597382546 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f25001f5870 con 0x7f24dc071390 2024-03-08T06:06:38.406 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-03-08T06:06:38.407 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T06:06:38.407 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-03-08T06:06:38.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.406+0000 7f2506400640 1 -- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] conn(0x7f24dc071390 msgr2=0x7f24dc0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.406+0000 7f2506400640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.53:6808/597382546,v1:172.21.15.53:6809/597382546] conn(0x7f24dc071390 0x7f24dc0737b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.406+0000 7f2506400640 1 -- 172.21.15.40:0/3487866500 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f24dc06b4d0 msgr2=0x7f24dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.406+0000 7f2506400640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f24dc06b4d0 0x7f24dc06d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f24e8036000 tx=0x7f24e8008030 comp rx=0 tx=0).stop 2024-03-08T06:06:38.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.406+0000 7f2506400640 1 -- 172.21.15.40:0/3487866500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2500071e90 msgr2=0x7f2500159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.406+0000 7f2506400640 1 --2- 172.21.15.40:0/3487866500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2500071e90 0x7f2500159480 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f24f000efc0 tx=0x7f24f000c490 comp rx=0 tx=0).stop 2024-03-08T06:06:38.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.407+0000 7f24ffe00640 1 -- 172.21.15.40:0/3487866500 reap_dead start 2024-03-08T06:06:38.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.407+0000 7f2506400640 1 -- 172.21.15.40:0/3487866500 shutdown_connections 2024-03-08T06:06:38.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.407+0000 7f2506400640 1 -- 172.21.15.40:0/3487866500 >> 172.21.15.40:0/3487866500 conn(0x7f250006ae30 msgr2=0x7f25000b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:38.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.407+0000 7f2506400640 1 -- 172.21.15.40:0/3487866500 shutdown_connections 2024-03-08T06:06:38.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.407+0000 7f2506400640 1 -- 172.21.15.40:0/3487866500 wait complete. 2024-03-08T06:06:38.425 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-08T06:06:38.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.649+0000 7f0646400640 1 Processor -- start 2024-03-08T06:06:38.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.649+0000 7f0646400640 1 -- start start 2024-03-08T06:06:38.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.651+0000 7f0646400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400cd730 0x7f06400cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.651+0000 7f0646400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06400cc690 con 0x7f0640145250 2024-03-08T06:06:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.651+0000 7f0646400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06400cc970 con 0x7f06400717d0 2024-03-08T06:06:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.651+0000 7f0646400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06400ccc50 con 0x7f06400cd730 2024-03-08T06:06:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.651+0000 7f063f400640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0640145250 0x7f064006eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50144/0 (socket says 172.21.15.40:50144) 2024-03-08T06:06:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.651+0000 7f063f400640 1 -- 172.21.15.40:0/110437907 learned_addr learned my addr 172.21.15.40:0/110437907 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:38.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.652+0000 7f063ea00640 1 --2- 172.21.15.40:0/110437907 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400cd730 0x7f06400cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.652+0000 7f063e000640 1 -- 172.21.15.40:0/110437907 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371736846 0 0) 0x7f06400cc690 con 0x7f0640145250 2024-03-08T06:06:38.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.652+0000 7f063e000640 1 -- 172.21.15.40:0/110437907 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0624003520 con 0x7f0640145250 2024-03-08T06:06:38.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.652+0000 7f063ea00640 1 -- 172.21.15.40:0/110437907 >> v1:172.21.15.53:6789/0 conn(0x7f06400717d0 legacy=0x7f064015c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.652+0000 7f063ea00640 1 -- 172.21.15.40:0/110437907 >> v1:172.21.15.40:6789/0 conn(0x7f0640145250 legacy=0x7f064006eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.652+0000 7f063ea00640 1 -- 172.21.15.40:0/110437907 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400ccf30 con 0x7f06400cd730 2024-03-08T06:06:38.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.652+0000 7f063e000640 1 -- 172.21.15.40:0/110437907 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981476497 0 0) 0x7f0624003520 con 0x7f0640145250 2024-03-08T06:06:38.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.654+0000 7f063ea00640 1 --2- 172.21.15.40:0/110437907 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400cd730 0x7f06400cc0b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f062c0097d0 tx=0x7f062c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a25c044f04d05f91 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:38.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.654+0000 7f063e000640 1 -- 172.21.15.40:0/110437907 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c034070 con 0x7f06400cd730 2024-03-08T06:06:38.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.654+0000 7f063e000640 1 -- 172.21.15.40:0/110437907 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c030800 con 0x7f06400cd730 2024-03-08T06:06:38.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.654+0000 7f063e000640 1 -- 172.21.15.40:0/110437907 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c030bb0 con 0x7f06400cd730 2024-03-08T06:06:38.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.655+0000 7f0646400640 1 -- 172.21.15.40:0/110437907 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400cd730 msgr2=0x7f06400cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.655+0000 7f0646400640 1 --2- 172.21.15.40:0/110437907 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400cd730 0x7f06400cc0b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f062c0097d0 tx=0x7f062c02b940 comp rx=0 tx=0).stop 2024-03-08T06:06:38.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.655+0000 7f0646400640 1 -- 172.21.15.40:0/110437907 shutdown_connections 2024-03-08T06:06:38.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.655+0000 7f0646400640 1 --2- 172.21.15.40:0/110437907 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400cd730 0x7f06400cc0b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.655+0000 7f0646400640 1 -- 172.21.15.40:0/110437907 >> 172.21.15.40:0/110437907 conn(0x7f064006ae30 msgr2=0x7f064006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.655+0000 7f0646400640 1 -- 172.21.15.40:0/110437907 shutdown_connections 2024-03-08T06:06:38.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.655+0000 7f0646400640 1 -- 172.21.15.40:0/110437907 wait complete. 2024-03-08T06:06:38.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.657+0000 7f0646400640 1 Processor -- start 2024-03-08T06:06:38.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.657+0000 7f0646400640 1 -- start start 2024-03-08T06:06:38.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.658+0000 7f0646400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400717d0 0x7f064013b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.658+0000 7f0646400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0640145250 0x7f064013f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.658+0000 7f0646400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064013c140 0x7f064015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.658+0000 7f0646400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06401457b0 con 0x7f0640145250 2024-03-08T06:06:38.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.658+0000 7f0646400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0640145630 con 0x7f064013c140 2024-03-08T06:06:38.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.658+0000 7f0646400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0640071c20 con 0x7f06400717d0 2024-03-08T06:06:38.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.658+0000 7f063ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0640145250 0x7f064013f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.658+0000 7f063ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0640145250 0x7f064013f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33918/0 (socket says 172.21.15.40:33918) 2024-03-08T06:06:38.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.658+0000 7f063ea00640 1 -- 172.21.15.40:0/1535715022 learned_addr learned my addr 172.21.15.40:0/1535715022 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.659+0000 7f063f400640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400717d0 0x7f064013b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.659+0000 7f063ea00640 1 -- 172.21.15.40:0/1535715022 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400717d0 msgr2=0x7f064013b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.659+0000 7f063ea00640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400717d0 0x7f064013b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.659+0000 7f063ea00640 1 -- 172.21.15.40:0/1535715022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064013c140 msgr2=0x7f064015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.659+0000 7f063fe00640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064013c140 0x7f064015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.659+0000 7f063ea00640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064013c140 0x7f064015a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.659+0000 7f063ea00640 1 -- 172.21.15.40:0/1535715022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064013fca0 con 0x7f0640145250 2024-03-08T06:06:38.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.659+0000 7f063f400640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400717d0 0x7f064013b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:38.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.659+0000 7f063ea00640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0640145250 0x7f064013f0c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f062c0098f0 tx=0x7f062c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:38.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.660+0000 7f0637400640 1 -- 172.21.15.40:0/1535715022 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c047070 con 0x7f0640145250 2024-03-08T06:06:38.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.660+0000 7f0637400640 1 -- 172.21.15.40:0/1535715022 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c030600 con 0x7f0640145250 2024-03-08T06:06:38.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.660+0000 7f0637400640 1 -- 172.21.15.40:0/1535715022 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c034050 con 0x7f0640145250 2024-03-08T06:06:38.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.660+0000 7f0646400640 1 -- 172.21.15.40:0/1535715022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06400b2bf0 con 0x7f0640145250 2024-03-08T06:06:38.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.660+0000 7f0646400640 1 -- 172.21.15.40:0/1535715022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06400b4ae0 con 0x7f0640145250 2024-03-08T06:06:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.661+0000 7f0646400640 1 -- 172.21.15.40:0/1535715022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0640067cb0 con 0x7f0640145250 2024-03-08T06:06:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.662+0000 7f0637400640 1 -- 172.21.15.40:0/1535715022 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f062c037070 con 0x7f0640145250 2024-03-08T06:06:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.663+0000 7f0637400640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f062006b4d0 0x7f062006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.663+0000 7f0637400640 1 -- 172.21.15.40:0/1535715022 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f062c0ab630 con 0x7f0640145250 2024-03-08T06:06:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.663+0000 7f063f400640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f062006b4d0 0x7f062006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.664+0000 7f063f400640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f062006b4d0 0x7f062006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f06300011e0 tx=0x7f0630009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:38.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.667+0000 7f0637400640 1 -- 172.21.15.40:0/1535715022 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f064013fca0 con 0x7f0640145250 2024-03-08T06:06:38.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.865+0000 7f0646400640 1 -- 172.21.15.40:0/1535715022 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f0640143530 con 0x7f062006b4d0 2024-03-08T06:06:38.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.866+0000 7f0637400640 1 -- 172.21.15.40:0/1535715022 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0640143530 con 0x7f062006b4d0 2024-03-08T06:06:38.866 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2024-03-08T06:06:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 -- 172.21.15.40:0/1535715022 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f062006b4d0 msgr2=0x7f062006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f062006b4d0 0x7f062006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f06300011e0 tx=0x7f0630009040 comp rx=0 tx=0).stop 2024-03-08T06:06:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 -- 172.21.15.40:0/1535715022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0640145250 msgr2=0x7f064013f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0640145250 0x7f064013f0c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f062c0098f0 tx=0x7f062c0046c0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 -- 172.21.15.40:0/1535715022 shutdown_connections 2024-03-08T06:06:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f062006b4d0 0x7f062006d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f064013c140 0x7f064015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0640145250 0x7f064013f0c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 --2- 172.21.15.40:0/1535715022 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400717d0 0x7f064013b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:38.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 -- 172.21.15.40:0/1535715022 >> 172.21.15.40:0/1535715022 conn(0x7f064006ae30 msgr2=0x7f064006e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:38.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 -- 172.21.15.40:0/1535715022 shutdown_connections 2024-03-08T06:06:38.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:38.869+0000 7f0646400640 1 -- 172.21.15.40:0/1535715022 wait complete. 2024-03-08T06:06:38.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:06:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.110+0000 7f32d3e00640 1 Processor -- start 2024-03-08T06:06:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.111+0000 7f32d3e00640 1 -- start start 2024-03-08T06:06:39.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.112+0000 7f32d3e00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f32d415df20 0x7f32d40ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:39.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.112+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32d40b6bd0 0x7f32d4162400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.112+0000 7f32d3e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d415e300 con 0x7f32d40d0aa0 2024-03-08T06:06:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.112+0000 7f32d3e00640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d40d3990 con 0x7f32d415df20 2024-03-08T06:06:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.112+0000 7f32d3e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d4162940 con 0x7f32d40b6bd0 2024-03-08T06:06:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.113+0000 7f32d2000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32d40b6bd0 0x7f32d4162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.113+0000 7f32d2a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f32d415df20 0x7f32d40ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:39.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.113+0000 7f32d2a00640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f32d415df20 0x7f32d40ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.40:53958/0 (socket says 172.21.15.40:53958) 2024-03-08T06:06:39.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.113+0000 7f32d2000640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32d40b6bd0 0x7f32d4162400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.40:60152/0 (socket says 172.21.15.40:60152) 2024-03-08T06:06:39.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.113+0000 7f32d2a00640 1 -- 172.21.15.40:0/3295877350 learned_addr learned my addr 172.21.15.40:0/3295877350 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:39.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.114+0000 7f32d2a00640 1 -- 172.21.15.40:0/3295877350 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32d40b6bd0 msgr2=0x7f32d4162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.114+0000 7f32d2a00640 1 --2- 172.21.15.40:0/3295877350 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32d40b6bd0 0x7f32d4162400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.114+0000 7f32d2a00640 1 -- 172.21.15.40:0/3295877350 >> v1:172.21.15.53:6789/0 conn(0x7f32d40d0aa0 legacy=0x7f32d40b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.114+0000 7f32d2a00640 1 -- 172.21.15.40:0/3295877350 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d40cf030 con 0x7f32d415df20 2024-03-08T06:06:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.116+0000 7f32d2a00640 1 --2- 172.21.15.40:0/3295877350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d415df20 0x7f32d40ce9e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009870 tx=0x7f32c402ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec99a5bf044036a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.116+0000 7f32d1600640 1 -- 172.21.15.40:0/3295877350 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c402f840 con 0x7f32d415df20 2024-03-08T06:06:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.116+0000 7f32d1600640 1 -- 172.21.15.40:0/3295877350 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c402f9e0 con 0x7f32d415df20 2024-03-08T06:06:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.116+0000 7f32d1600640 1 -- 172.21.15.40:0/3295877350 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c402fdb0 con 0x7f32d415df20 2024-03-08T06:06:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.117+0000 7f32d3e00640 1 -- 172.21.15.40:0/3295877350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d415df20 msgr2=0x7f32d40ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.117+0000 7f32d3e00640 1 --2- 172.21.15.40:0/3295877350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d415df20 0x7f32d40ce9e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009870 tx=0x7f32c402ba20 comp rx=0 tx=0).stop 2024-03-08T06:06:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.118+0000 7f32d3e00640 1 -- 172.21.15.40:0/3295877350 shutdown_connections 2024-03-08T06:06:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.118+0000 7f32d3e00640 1 --2- 172.21.15.40:0/3295877350 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32d40b6bd0 0x7f32d4162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.118+0000 7f32d3e00640 1 --2- 172.21.15.40:0/3295877350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d415df20 0x7f32d40ce9e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.118+0000 7f32d3e00640 1 -- 172.21.15.40:0/3295877350 >> 172.21.15.40:0/3295877350 conn(0x7f32d406c740 msgr2=0x7f32d406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.118+0000 7f32d3e00640 1 -- 172.21.15.40:0/3295877350 shutdown_connections 2024-03-08T06:06:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.118+0000 7f32d3e00640 1 -- 172.21.15.40:0/3295877350 wait complete. 2024-03-08T06:06:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.120+0000 7f32d3e00640 1 Processor -- start 2024-03-08T06:06:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.120+0000 7f32d3e00640 1 -- start start 2024-03-08T06:06:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.121+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32d40b6bd0 0x7f32d40b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.121+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d40d0aa0 0x7f32d40c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.121+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d415df20 0x7f32d40c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.121+0000 7f32d3e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d40d2180 con 0x7f32d40d0aa0 2024-03-08T06:06:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.121+0000 7f32d3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d40d1ea0 con 0x7f32d415df20 2024-03-08T06:06:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.121+0000 7f32d3e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d40d2460 con 0x7f32d40b6bd0 2024-03-08T06:06:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.122+0000 7f32d2000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d40d0aa0 0x7f32d40c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.122+0000 7f32d2000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d40d0aa0 0x7f32d40c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33922/0 (socket says 172.21.15.40:33922) 2024-03-08T06:06:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.122+0000 7f32d2000640 1 -- 172.21.15.40:0/4020899298 learned_addr learned my addr 172.21.15.40:0/4020899298 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.122+0000 7f32d2a00640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32d40b6bd0 0x7f32d40b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.122+0000 7f32d3400640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d415df20 0x7f32d40c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.122+0000 7f32d2000640 1 -- 172.21.15.40:0/4020899298 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32d40b6bd0 msgr2=0x7f32d40b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.122+0000 7f32d2000640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32d40b6bd0 0x7f32d40b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.122+0000 7f32d2000640 1 -- 172.21.15.40:0/4020899298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d415df20 msgr2=0x7f32d40c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.122+0000 7f32d2000640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d415df20 0x7f32d40c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.122+0000 7f32d2000640 1 -- 172.21.15.40:0/4020899298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d415e620 con 0x7f32d40d0aa0 2024-03-08T06:06:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.122+0000 7f32d3400640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d415df20 0x7f32d40c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.123+0000 7f32d2000640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d40d0aa0 0x7f32d40c86f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00e9b0 tx=0x7f32bc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.123+0000 7f32cb400640 1 -- 172.21.15.40:0/4020899298 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc017070 con 0x7f32d40d0aa0 2024-03-08T06:06:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.123+0000 7f32cb400640 1 -- 172.21.15.40:0/4020899298 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32bc00c9a0 con 0x7f32d40d0aa0 2024-03-08T06:06:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.123+0000 7f32cb400640 1 -- 172.21.15.40:0/4020899298 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00cd90 con 0x7f32d40d0aa0 2024-03-08T06:06:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.123+0000 7f32d3e00640 1 -- 172.21.15.40:0/4020899298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d415e8c0 con 0x7f32d40d0aa0 2024-03-08T06:06:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.123+0000 7f32d3e00640 1 -- 172.21.15.40:0/4020899298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d415ee80 con 0x7f32d40d0aa0 2024-03-08T06:06:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.124+0000 7f32d3e00640 1 -- 172.21.15.40:0/4020899298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d4075100 con 0x7f32d40d0aa0 2024-03-08T06:06:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.125+0000 7f32cb400640 1 -- 172.21.15.40:0/4020899298 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32bc01a070 con 0x7f32d40d0aa0 2024-03-08T06:06:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.126+0000 7f32cb400640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f32b006b4d0 0x7f32b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.126+0000 7f32cb400640 1 -- 172.21.15.40:0/4020899298 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f32bc089f50 con 0x7f32d40d0aa0 2024-03-08T06:06:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.127+0000 7f32d2a00640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f32b006b4d0 0x7f32b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.128+0000 7f32d2a00640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f32b006b4d0 0x7f32b006d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f32c4036000 tx=0x7f32c4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.132+0000 7f32cb400640 1 -- 172.21.15.40:0/4020899298 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32bc0590c0 con 0x7f32d40d0aa0 2024-03-08T06:06:39.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.330+0000 7f32d3e00640 1 -- 172.21.15.40:0/4020899298 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32d40d16e0 con 0x7f32b006b4d0 2024-03-08T06:06:39.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.331+0000 7f32cb400640 1 -- 172.21.15.40:0/4020899298 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13353 (secure 0 0 0) 0x7f32d40d16e0 con 0x7f32b006b4d0 2024-03-08T06:06:39.332 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:39.332 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-03-08T06:06:39.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 -- 172.21.15.40:0/4020899298 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f32b006b4d0 msgr2=0x7f32b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:39.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f32b006b4d0 0x7f32b006d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f32c4036000 tx=0x7f32c4008030 comp rx=0 tx=0).stop 2024-03-08T06:06:39.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 -- 172.21.15.40:0/4020899298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d40d0aa0 msgr2=0x7f32d40c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:39.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d40d0aa0 0x7f32d40c86f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00e9b0 tx=0x7f32bc00ee70 comp rx=0 tx=0).stop 2024-03-08T06:06:39.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 -- 172.21.15.40:0/4020899298 shutdown_connections 2024-03-08T06:06:39.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f32b006b4d0 0x7f32b006d990 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f32c4036000 tx=0x7f32c4008030 comp rx=0 tx=0).stop 2024-03-08T06:06:39.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32d415df20 0x7f32d40c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:39.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32d40d0aa0 0x7f32d40c86f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:39.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 --2- 172.21.15.40:0/4020899298 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32d40b6bd0 0x7f32d40b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:39.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 -- 172.21.15.40:0/4020899298 >> 172.21.15.40:0/4020899298 conn(0x7f32d406c740 msgr2=0x7f32d40b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:39.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 -- 172.21.15.40:0/4020899298 shutdown_connections 2024-03-08T06:06:39.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:39.334+0000 7f32d3e00640 1 -- 172.21.15.40:0/4020899298 wait complete. 2024-03-08T06:06:39.352 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-03-08T06:06:37.538713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262300,"kb_used_data":4520,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754712420,"statfs":{"total":773094113280,"available":772825518080,"internally_reserved":0,"allocated":4628480,"data_stored":2532224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.228093"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":200,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-08T06:06:27.859184+0000","last_change":"2024-03-08T06:03:57.473597+0000","last_active":"2024-03-08T06:06:27.859184+0000","last_peered":"2024-03-08T06:06:27.859184+0000","last_clean":"2024-03-08T06:06:27.859184+0000","last_became_active":"2024-03-08T06:03:57.468593+0000","last_became_peered":"2024-03-08T06:03:57.468593+0000","last_unstale":"2024-03-08T06:06:27.859184+0000","last_undegraded":"2024-03-08T06:06:27.859184+0000","last_fullsized":"2024-03-08T06:06:27.859184+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T06:03:56.214441+0000","last_clean_scrub_stamp":"2024-03-08T06:03:56.214441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:45:28.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1024000,"data_stored":759304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35760,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336080,"statfs":{"total":96636764160,"available":96600145920,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35760,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336080,"statfs":{"total":96636764160,"available":96600145920,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":759304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:06:39.355 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-08T06:03:56.214441+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=8, tm_hour=6, tm_min=3, tm_sec=56, tm_wday=4, tm_yday=68, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=8, tm_hour=6, tm_min=6, tm_sec=31, tm_wday=4, tm_yday=68, tm_isdst=0) 2024-03-08T06:06:39.355 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-08T06:06:58.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:06:58.640 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:06:59.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T06:06:59.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.572+0000 7f8ef8e00640 1 Processor -- start 2024-03-08T06:06:59.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.572+0000 7f8ef8e00640 1 -- start start 2024-03-08T06:06:59.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.574+0000 7f8ef8e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ef40c83b0 0x7f8ef40d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:59.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.574+0000 7f8ef8e00640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef40cdf90 con 0x7f8ef415f200 2024-03-08T06:06:59.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.574+0000 7f8ef8e00640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef40c88b0 con 0x7f8ef40cc100 2024-03-08T06:06:59.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.574+0000 7f8ef8e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40c73b0 con 0x7f8ef40c83b0 2024-03-08T06:06:59.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.574+0000 7f8ef1e00640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8ef415f200 0x7f8ef40c7840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49846/0 (socket says 172.21.15.40:49846) 2024-03-08T06:06:59.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.574+0000 7f8ef1e00640 1 -- 172.21.15.40:0/544605342 learned_addr learned my addr 172.21.15.40:0/544605342 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-03-08T06:06:59.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef1400640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ef40c83b0 0x7f8ef40d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:59.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361243465 0 0) 0x7f8ef40cdf90 con 0x7f8ef415f200 2024-03-08T06:06:59.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ed8003520 con 0x7f8ef415f200 2024-03-08T06:06:59.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1732382956 0 0) 0x7f8ed8003520 con 0x7f8ef415f200 2024-03-08T06:06:59.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ef40c83b0 msgr2=0x7f8ef40d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:59.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ef40c83b0 0x7f8ef40d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:59.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 >> v1:172.21.15.53:6789/0 conn(0x7f8ef40cc100 legacy=0x7f8ef40b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:59.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40c6a80 con 0x7f8ef415f200 2024-03-08T06:06:59.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 695796805 0 0) 0x7f8ee40019f0 con 0x7f8ef415f200 2024-03-08T06:06:59.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef1400640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ef40c83b0 0x7f8ef40d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T06:06:59.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 >> v1:172.21.15.40:6789/0 conn(0x7f8ef415f200 legacy=0x7f8ef40c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:59.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed80039e0 0x7f8ed8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:59.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ef40c83b0 0x7f8ed8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:59.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed8004d40 0x7f8ed80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:59.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40c73b0 con 0x7f8ed80039e0 2024-03-08T06:06:59.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed800bad0 con 0x7f8ed8004d40 2024-03-08T06:06:59.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.575+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed800bcd0 con 0x7f8ef40c83b0 2024-03-08T06:06:59.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.576+0000 7f8ef1e00640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed80039e0 0x7f8ed8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:59.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.576+0000 7f8ef1400640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ef40c83b0 0x7f8ed8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:59.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.576+0000 7f8ef2800640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed8004d40 0x7f8ed80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:59.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.576+0000 7f8ef1e00640 1 -- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ef40c83b0 msgr2=0x7f8ed8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:59.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.576+0000 7f8ef1e00640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ef40c83b0 0x7f8ed8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:59.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.576+0000 7f8ef1e00640 1 -- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed8004d40 msgr2=0x7f8ed80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:06:59.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.576+0000 7f8ef1e00640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed8004d40 0x7f8ed80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:59.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.576+0000 7f8ef1e00640 1 -- 172.21.15.40:0/544605342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed8003520 con 0x7f8ed80039e0 2024-03-08T06:06:59.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.577+0000 7f8ef1e00640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed80039e0 0x7f8ed8003e00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4002230 tx=0x7f8ed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e62e6268493ce5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:59.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.577+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400a4a0 con 0x7f8ed80039e0 2024-03-08T06:06:59.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.578+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee4034650 con 0x7f8ed80039e0 2024-03-08T06:06:59.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.578+0000 7f8ef0a00640 1 -- 172.21.15.40:0/544605342 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4034a40 con 0x7f8ed80039e0 2024-03-08T06:06:59.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.578+0000 7f8ef8e00640 1 -- 172.21.15.40:0/544605342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed80039e0 msgr2=0x7f8ed8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:59.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.578+0000 7f8ef8e00640 1 --2- 172.21.15.40:0/544605342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed80039e0 0x7f8ed8003e00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4002230 tx=0x7f8ed8005830 comp rx=0 tx=0).stop 2024-03-08T06:06:59.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.578+0000 7f8ef2800640 1 -- 172.21.15.40:0/544605342 reap_dead start 2024-03-08T06:06:59.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.578+0000 7f8ef8e00640 1 -- 172.21.15.40:0/544605342 shutdown_connections 2024-03-08T06:06:59.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.578+0000 7f8ef8e00640 1 -- 172.21.15.40:0/544605342 >> 172.21.15.40:0/544605342 conn(0x7f8ef40d1a90 msgr2=0x7f8ef40d1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:59.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.579+0000 7f8ef8e00640 1 -- 172.21.15.40:0/544605342 shutdown_connections 2024-03-08T06:06:59.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.579+0000 7f8ef8e00640 1 -- 172.21.15.40:0/544605342 wait complete. 2024-03-08T06:06:59.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.580+0000 7f8ef8e00640 1 Processor -- start 2024-03-08T06:06:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.580+0000 7f8ef8e00640 1 -- start start 2024-03-08T06:06:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.581+0000 7f8ef8e00640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ed80039e0 0x7f8ef414dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.581+0000 7f8ef8e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef414e1d0 0x7f8ef4150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.581+0000 7f8ef8e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef4150c40 0x7f8ef40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.581+0000 7f8ef8e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40c7f50 con 0x7f8ef4150c40 2024-03-08T06:06:59.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.581+0000 7f8ef8e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40c7040 con 0x7f8ef414e1d0 2024-03-08T06:06:59.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.581+0000 7f8ef8e00640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40c80d0 con 0x7f8ed80039e0 2024-03-08T06:06:59.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef2800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef4150c40 0x7f8ef40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:59.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef2800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef4150c40 0x7f8ef40b60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44874/0 (socket says 172.21.15.40:44874) 2024-03-08T06:06:59.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef2800640 1 -- 172.21.15.40:0/189358829 learned_addr learned my addr 172.21.15.40:0/189358829 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:06:59.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef1e00640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ed80039e0 0x7f8ef414dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef1400640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef414e1d0 0x7f8ef4150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef2800640 1 -- 172.21.15.40:0/189358829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ed80039e0 msgr2=0x7f8ef414dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef2800640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ed80039e0 0x7f8ef414dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef2800640 1 -- 172.21.15.40:0/189358829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef414e1d0 msgr2=0x7f8ef4150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef2800640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef414e1d0 0x7f8ef4150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef2800640 1 -- 172.21.15.40:0/189358829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40b6e80 con 0x7f8ef4150c40 2024-03-08T06:06:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef1e00640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ed80039e0 0x7f8ef414dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:06:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.582+0000 7f8ef2800640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef4150c40 0x7f8ef40b60e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4068090 tx=0x7f8eec00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.583+0000 7f8eea000640 1 -- 172.21.15.40:0/189358829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec018070 con 0x7f8ef4150c40 2024-03-08T06:06:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.583+0000 7f8eea000640 1 -- 172.21.15.40:0/189358829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec0035f0 con 0x7f8ef4150c40 2024-03-08T06:06:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.583+0000 7f8eea000640 1 -- 172.21.15.40:0/189358829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec003a20 con 0x7f8ef4150c40 2024-03-08T06:06:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.583+0000 7f8ef8e00640 1 -- 172.21.15.40:0/189358829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef40b7160 con 0x7f8ef4150c40 2024-03-08T06:06:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.583+0000 7f8ef8e00640 1 -- 172.21.15.40:0/189358829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef40b7720 con 0x7f8ef4150c40 2024-03-08T06:06:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.584+0000 7f8ef8e00640 1 -- 172.21.15.40:0/189358829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef40683c0 con 0x7f8ef4150c40 2024-03-08T06:06:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.585+0000 7f8eea000640 1 -- 172.21.15.40:0/189358829 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8eec01b070 con 0x7f8ef4150c40 2024-03-08T06:06:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.586+0000 7f8eea000640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8ed406b500 0x7f8ed406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:06:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.586+0000 7f8eea000640 1 -- 172.21.15.40:0/189358829 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8eec0895a0 con 0x7f8ef4150c40 2024-03-08T06:06:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.586+0000 7f8ef1e00640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8ed406b500 0x7f8ed406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:06:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.587+0000 7f8ef1e00640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8ed406b500 0x7f8ed406d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4034000 tx=0x7f8ee402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:06:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.595+0000 7f8eea000640 1 -- 172.21.15.40:0/189358829 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8eec058680 con 0x7f8ef4150c40 2024-03-08T06:06:59.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.791+0000 7f8ef8e00640 1 -- 172.21.15.40:0/189358829 --> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ef415d8b0 con 0x7f8ed406b500 2024-03-08T06:06:59.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.792+0000 7f8eea000640 1 -- 172.21.15.40:0/189358829 <== mgr.4106 v2:172.21.15.40:6832/448564530 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13359 (secure 0 0 0) 0x7f8ef415d8b0 con 0x7f8ed406b500 2024-03-08T06:06:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:06:59.794 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-03-08T06:06:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.794+0000 7f8ef8e00640 1 -- 172.21.15.40:0/189358829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8ed406b500 msgr2=0x7f8ed406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.794+0000 7f8ef8e00640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8ed406b500 0x7f8ed406d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4034000 tx=0x7f8ee402f000 comp rx=0 tx=0).stop 2024-03-08T06:06:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.794+0000 7f8ef8e00640 1 -- 172.21.15.40:0/189358829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef4150c40 msgr2=0x7f8ef40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:06:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.794+0000 7f8ef8e00640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef4150c40 0x7f8ef40b60e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4068090 tx=0x7f8eec00d890 comp rx=0 tx=0).stop 2024-03-08T06:06:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.794+0000 7f8ef8e00640 1 -- 172.21.15.40:0/189358829 shutdown_connections 2024-03-08T06:06:59.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.795+0000 7f8ef8e00640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8ed406b500 0x7f8ed406d9c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:59.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.795+0000 7f8ef8e00640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef4150c40 0x7f8ef40b60e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:59.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.795+0000 7f8ef8e00640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ef414e1d0 0x7f8ef4150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:59.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.795+0000 7f8ef8e00640 1 --2- 172.21.15.40:0/189358829 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ed80039e0 0x7f8ef414dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:06:59.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.795+0000 7f8ef8e00640 1 -- 172.21.15.40:0/189358829 >> 172.21.15.40:0/189358829 conn(0x7f8ef40d1a90 msgr2=0x7f8ef40b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:06:59.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.795+0000 7f8ef8e00640 1 -- 172.21.15.40:0/189358829 shutdown_connections 2024-03-08T06:06:59.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:06:59.795+0000 7f8ef8e00640 1 -- 172.21.15.40:0/189358829 wait complete. 2024-03-08T06:06:59.816 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-03-08T06:06:59.545079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262288,"kb_used_data":4520,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754712432,"statfs":{"total":773094113280,"available":772825530368,"internally_reserved":0,"allocated":4628480,"data_stored":2532224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003440"},"pg_stats":[{"pgid":"1.0","version":"37'119","reported_seq":211,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-08T06:06:39.438853+0000","last_change":"2024-03-08T06:06:39.438929+0000","last_active":"2024-03-08T06:06:39.438853+0000","last_peered":"2024-03-08T06:06:39.438853+0000","last_clean":"2024-03-08T06:06:39.438853+0000","last_became_active":"2024-03-08T06:03:57.468593+0000","last_became_peered":"2024-03-08T06:03:57.468593+0000","last_unstale":"2024-03-08T06:06:39.438853+0000","last_undegraded":"2024-03-08T06:06:39.438853+0000","last_fullsized":"2024-03-08T06:06:39.438853+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'119","last_scrub_stamp":"2024-03-08T06:06:39.438643+0000","last_deep_scrub":"37'119","last_deep_scrub_stamp":"2024-03-08T06:06:39.438643+0000","last_clean_scrub_stamp":"2024-03-08T06:06:39.438643+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:42:08.958650+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1024000,"data_stored":759304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35760,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336080,"statfs":{"total":96636764160,"available":96600145920,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35760,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336080,"statfs":{"total":96636764160,"available":96600145920,"internally_reserved":0,"allocated":430080,"data_stored":168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1024000,"data_stored":759304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T06:06:59.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-08T06:07:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.006+0000 7f8ca1800640 1 Processor -- start 2024-03-08T06:07:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.007+0000 7f8ca1800640 1 -- start start 2024-03-08T06:07:00.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.008+0000 7f8ca1800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c9c09a090 0x7f8c9c0cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:07:00.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.008+0000 7f8ca1800640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8c9c09a470 0x7f8c9c0cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:07:00.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.008+0000 7f8ca1800640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9c0ce0b0 con 0x7f8c9c06d230 2024-03-08T06:07:00.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.009+0000 7f8ca1800640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c0d0ec0 con 0x7f8c9c09a090 2024-03-08T06:07:00.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.009+0000 7f8ca1800640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c0cd010 con 0x7f8c9c09a470 2024-03-08T06:07:00.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.009+0000 7f8c9aa00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c9c09a090 0x7f8c9c0cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:07:00.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.009+0000 7f8c9aa00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c9c09a090 0x7f8c9c0cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44876/0 (socket says 172.21.15.40:44876) 2024-03-08T06:07:00.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.009+0000 7f8c9aa00640 1 -- 172.21.15.40:0/116391280 learned_addr learned my addr 172.21.15.40:0/116391280 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:07:00.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.009+0000 7f8c9a000640 1 --2- 172.21.15.40:0/116391280 >> v2:172.21.15.53:3300/0 conn(0x7f8c9c09a470 0x7f8c9c0cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:07:00.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.009+0000 7f8c9aa00640 1 -- 172.21.15.40:0/116391280 >> v2:172.21.15.53:3300/0 conn(0x7f8c9c09a470 msgr2=0x7f8c9c0cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:07:00.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.009+0000 7f8c9aa00640 1 --2- 172.21.15.40:0/116391280 >> v2:172.21.15.53:3300/0 conn(0x7f8c9c09a470 0x7f8c9c0cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.009+0000 7f8c9aa00640 1 -- 172.21.15.40:0/116391280 >> v1:172.21.15.40:6789/0 conn(0x7f8c9c06d230 legacy=0x7f8c9c06d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.009+0000 7f8c9aa00640 1 -- 172.21.15.40:0/116391280 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c0cd2f0 con 0x7f8c9c09a090 2024-03-08T06:07:00.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.009+0000 7f8c9a000640 1 --2- 172.21.15.40:0/116391280 >> v2:172.21.15.53:3300/0 conn(0x7f8c9c09a470 0x7f8c9c0cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T06:07:00.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.011+0000 7f8c9aa00640 1 --2- 172.21.15.40:0/116391280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c09a090 0x7f8c9c0cca10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c072f20 tx=0x7f8c8c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3998269f8368f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:07:00.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.011+0000 7f8c99600640 1 -- 172.21.15.40:0/116391280 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c02cb10 con 0x7f8c9c09a090 2024-03-08T06:07:00.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.011+0000 7f8c99600640 1 -- 172.21.15.40:0/116391280 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c02ccb0 con 0x7f8c9c09a090 2024-03-08T06:07:00.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.011+0000 7f8c99600640 1 -- 172.21.15.40:0/116391280 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c02ce60 con 0x7f8c9c09a090 2024-03-08T06:07:00.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.012+0000 7f8ca1800640 1 -- 172.21.15.40:0/116391280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c09a090 msgr2=0x7f8c9c0cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.012+0000 7f8ca1800640 1 --2- 172.21.15.40:0/116391280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c09a090 0x7f8c9c0cca10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c072f20 tx=0x7f8c8c02bf70 comp rx=0 tx=0).stop 2024-03-08T06:07:00.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.012+0000 7f8ca1800640 1 -- 172.21.15.40:0/116391280 shutdown_connections 2024-03-08T06:07:00.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.012+0000 7f8ca1800640 1 --2- 172.21.15.40:0/116391280 >> v2:172.21.15.53:3300/0 conn(0x7f8c9c09a470 0x7f8c9c0cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.012+0000 7f8ca1800640 1 --2- 172.21.15.40:0/116391280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c09a090 0x7f8c9c0cca10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.012+0000 7f8ca1800640 1 -- 172.21.15.40:0/116391280 >> 172.21.15.40:0/116391280 conn(0x7f8c9c0b3710 msgr2=0x7f8c9c0b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:07:00.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.012+0000 7f8ca1800640 1 -- 172.21.15.40:0/116391280 shutdown_connections 2024-03-08T06:07:00.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.012+0000 7f8ca1800640 1 -- 172.21.15.40:0/116391280 wait complete. 2024-03-08T06:07:00.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.014+0000 7f8ca1800640 1 Processor -- start 2024-03-08T06:07:00.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.014+0000 7f8ca1800640 1 -- start start 2024-03-08T06:07:00.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8ca1800640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c9c06d230 0x7f8c9c1570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:07:00.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8ca1800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c09a090 0x7f8c9c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:07:00.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8ca1800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c9c09a470 0x7f8c9c159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:07:00.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8ca1800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0cdb90 con 0x7f8c9c09a090 2024-03-08T06:07:00.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8ca1800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0cd8b0 con 0x7f8c9c09a470 2024-03-08T06:07:00.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8ca1800640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0cfef0 con 0x7f8c9c06d230 2024-03-08T06:07:00.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8c9a000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c09a090 0x7f8c9c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:07:00.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8c9a000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c09a090 0x7f8c9c15bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44892/0 (socket says 172.21.15.40:44892) 2024-03-08T06:07:00.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8c9a000640 1 -- 172.21.15.40:0/3519579224 learned_addr learned my addr 172.21.15.40:0/3519579224 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:07:00.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8c9aa00640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c9c06d230 0x7f8c9c1570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:07:00.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8c9b400640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c9c09a470 0x7f8c9c159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:07:00.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8c9a000640 1 -- 172.21.15.40:0/3519579224 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c9c06d230 msgr2=0x7f8c9c1570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8c9a000640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c9c06d230 0x7f8c9c1570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8c9a000640 1 -- 172.21.15.40:0/3519579224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c9c09a470 msgr2=0x7f8c9c159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8c9a000640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c9c09a470 0x7f8c9c159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.015+0000 7f8c9a000640 1 -- 172.21.15.40:0/3519579224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c0c8e20 con 0x7f8c9c09a090 2024-03-08T06:07:00.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.016+0000 7f8c9a000640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c09a090 0x7f8c9c15bfd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800c7f0 tx=0x7f8c8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:07:00.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.016+0000 7f8c92a00640 1 -- 172.21.15.40:0/3519579224 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88015070 con 0x7f8c9c09a090 2024-03-08T06:07:00.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.016+0000 7f8c92a00640 1 -- 172.21.15.40:0/3519579224 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c88007a30 con 0x7f8c9c09a090 2024-03-08T06:07:00.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.016+0000 7f8c92a00640 1 -- 172.21.15.40:0/3519579224 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88007e00 con 0x7f8c9c09a090 2024-03-08T06:07:00.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.016+0000 7f8ca1800640 1 -- 172.21.15.40:0/3519579224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c0c9100 con 0x7f8c9c09a090 2024-03-08T06:07:00.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.016+0000 7f8ca1800640 1 -- 172.21.15.40:0/3519579224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c157630 con 0x7f8c9c09a090 2024-03-08T06:07:00.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.017+0000 7f8ca1800640 1 -- 172.21.15.40:0/3519579224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c0683c0 con 0x7f8c9c09a090 2024-03-08T06:07:00.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.018+0000 7f8c92a00640 1 -- 172.21.15.40:0/3519579224 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c88018070 con 0x7f8c9c09a090 2024-03-08T06:07:00.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.019+0000 7f8c92a00640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8c7c06b520 0x7f8c7c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:07:00.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.019+0000 7f8c92a00640 1 -- 172.21.15.40:0/3519579224 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8c88089e40 con 0x7f8c9c09a090 2024-03-08T06:07:00.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.020+0000 7f8c9aa00640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8c7c06b520 0x7f8c7c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:07:00.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.021+0000 7f8c9aa00640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8c7c06b520 0x7f8c7c06d9e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c035000 tx=0x7f8c8c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:07:00.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.024+0000 7f8c92a00640 1 -- 172.21.15.40:0/3519579224 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c88059030 con 0x7f8c9c09a090 2024-03-08T06:07:00.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.225+0000 7f8ca1800640 1 -- 172.21.15.40:0/3519579224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f8c9c0cca80 con 0x7f8c9c09a090 2024-03-08T06:07:00.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.230+0000 7f8c92a00640 1 -- 172.21.15.40:0/3519579224 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8c88058930 con 0x7f8c9c09a090 2024-03-08T06:07:00.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.231+0000 7f8c92a00640 1 -- 172.21.15.40:0/3519579224 <== mon.0 v2:172.21.15.40:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8c8800ce40 con 0x7f8c9c09a090 2024-03-08T06:07:00.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.233+0000 7f8ca1800640 1 -- 172.21.15.40:0/3519579224 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8c7c06b520 msgr2=0x7f8c7c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.233+0000 7f8ca1800640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8c7c06b520 0x7f8c7c06d9e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c035000 tx=0x7f8c8c030000 comp rx=0 tx=0).stop 2024-03-08T06:07:00.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.233+0000 7f8ca1800640 1 -- 172.21.15.40:0/3519579224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c09a090 msgr2=0x7f8c9c15bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.233+0000 7f8ca1800640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c09a090 0x7f8c9c15bfd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800c7f0 tx=0x7f8c8800cbf0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.234+0000 7f8ca1800640 1 -- 172.21.15.40:0/3519579224 shutdown_connections 2024-03-08T06:07:00.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.234+0000 7f8ca1800640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f8c7c06b520 0x7f8c7c06d9e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.234+0000 7f8ca1800640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c9c09a470 0x7f8c9c159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.234+0000 7f8ca1800640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c09a090 0x7f8c9c15bfd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.234+0000 7f8ca1800640 1 --2- 172.21.15.40:0/3519579224 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c9c06d230 0x7f8c9c1570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.234+0000 7f8ca1800640 1 -- 172.21.15.40:0/3519579224 >> 172.21.15.40:0/3519579224 conn(0x7f8c9c0b3710 msgr2=0x7f8c9c0af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:07:00.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.234+0000 7f8ca1800640 1 -- 172.21.15.40:0/3519579224 shutdown_connections 2024-03-08T06:07:00.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.234+0000 7f8ca1800640 1 -- 172.21.15.40:0/3519579224 wait complete. 2024-03-08T06:07:00.255 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-08T06:07:00.256 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T06:07:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.448+0000 7f5e50400640 1 Processor -- start 2024-03-08T06:07:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.449+0000 7f5e50400640 1 -- start start 2024-03-08T06:07:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.450+0000 7f5e50400640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e481613b0 0x7f5e480d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:07:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.450+0000 7f5e50400640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e480d25b0 con 0x7f5e4806c3e0 2024-03-08T06:07:00.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.450+0000 7f5e50400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e480d2890 con 0x7f5e480b9500 2024-03-08T06:07:00.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.450+0000 7f5e50400640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e480d2b70 con 0x7f5e481613b0 2024-03-08T06:07:00.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.450+0000 7f5e4da00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e481613b0 0x7f5e480d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:07:00.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.450+0000 7f5e4da00640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e481613b0 0x7f5e480d3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44896/0 (socket says 172.21.15.40:44896) 2024-03-08T06:07:00.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.450+0000 7f5e4da00640 1 -- 172.21.15.40:0/3476497704 learned_addr learned my addr 172.21.15.40:0/3476497704 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:07:00.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.450+0000 7f5e4e400640 1 --1- 172.21.15.40:0/3476497704 >> v1:172.21.15.40:6789/0 conn(0x7f5e4806c3e0 0x7f5e480b6c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49872/0 (socket says 172.21.15.40:49872) 2024-03-08T06:07:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.451+0000 7f5e4da00640 1 -- 172.21.15.40:0/3476497704 >> v1:172.21.15.53:6789/0 conn(0x7f5e480b9500 legacy=0x7f5e480d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T06:07:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.451+0000 7f5e4da00640 1 -- 172.21.15.40:0/3476497704 >> v1:172.21.15.40:6789/0 conn(0x7f5e4806c3e0 legacy=0x7f5e480b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.451+0000 7f5e4da00640 1 -- 172.21.15.40:0/3476497704 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480d2e50 con 0x7f5e481613b0 2024-03-08T06:07:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.451+0000 7f5e47e00640 1 -- 172.21.15.40:0/3476497704 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294040361 0 0) 0x7f5e480d25b0 con 0x7f5e4806c3e0 2024-03-08T06:07:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.452+0000 7f5e4da00640 1 --2- 172.21.15.40:0/3476497704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e481613b0 0x7f5e480d3650 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0098a0 tx=0x7f5e3c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f420aa0348b9fa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:07:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.453+0000 7f5e47e00640 1 -- 172.21.15.40:0/3476497704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c02f840 con 0x7f5e481613b0 2024-03-08T06:07:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.453+0000 7f5e47e00640 1 -- 172.21.15.40:0/3476497704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5e3c02f9e0 con 0x7f5e481613b0 2024-03-08T06:07:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.453+0000 7f5e47e00640 1 -- 172.21.15.40:0/3476497704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c02fdd0 con 0x7f5e481613b0 2024-03-08T06:07:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.453+0000 7f5e50400640 1 -- 172.21.15.40:0/3476497704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e481613b0 msgr2=0x7f5e480d3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.453+0000 7f5e50400640 1 --2- 172.21.15.40:0/3476497704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e481613b0 0x7f5e480d3650 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0098a0 tx=0x7f5e3c02ba20 comp rx=0 tx=0).stop 2024-03-08T06:07:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.453+0000 7f5e50400640 1 -- 172.21.15.40:0/3476497704 shutdown_connections 2024-03-08T06:07:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.453+0000 7f5e50400640 1 --2- 172.21.15.40:0/3476497704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e481613b0 0x7f5e480d3650 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.453+0000 7f5e50400640 1 -- 172.21.15.40:0/3476497704 >> 172.21.15.40:0/3476497704 conn(0x7f5e480729b0 msgr2=0x7f5e480d3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:07:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.453+0000 7f5e50400640 1 -- 172.21.15.40:0/3476497704 shutdown_connections 2024-03-08T06:07:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.453+0000 7f5e50400640 1 -- 172.21.15.40:0/3476497704 wait complete. 2024-03-08T06:07:00.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.455+0000 7f5e50400640 1 Processor -- start 2024-03-08T06:07:00.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.455+0000 7f5e50400640 1 -- start start 2024-03-08T06:07:00.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.456+0000 7f5e50400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e4806c3e0 0x7f5e481f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:07:00.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.456+0000 7f5e50400640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e480b9500 0x7f5e480acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:07:00.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.456+0000 7f5e50400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e481613b0 0x7f5e480ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:07:00.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.456+0000 7f5e50400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4806c810 con 0x7f5e481613b0 2024-03-08T06:07:00.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.456+0000 7f5e50400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48161790 con 0x7f5e4806c3e0 2024-03-08T06:07:00.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.456+0000 7f5e50400640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4806c990 con 0x7f5e480b9500 2024-03-08T06:07:00.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.456+0000 7f5e4e400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e481613b0 0x7f5e480ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:07:00.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.457+0000 7f5e4e400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e481613b0 0x7f5e480ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44904/0 (socket says 172.21.15.40:44904) 2024-03-08T06:07:00.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.457+0000 7f5e4e400640 1 -- 172.21.15.40:0/2496962452 learned_addr learned my addr 172.21.15.40:0/2496962452 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-03-08T06:07:00.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.457+0000 7f5e4da00640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e4806c3e0 0x7f5e481f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:07:00.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.457+0000 7f5e4d000640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e480b9500 0x7f5e480acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:07:00.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.457+0000 7f5e4e400640 1 -- 172.21.15.40:0/2496962452 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e480b9500 msgr2=0x7f5e480acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.457+0000 7f5e4e400640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e480b9500 0x7f5e480acff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.457+0000 7f5e4e400640 1 -- 172.21.15.40:0/2496962452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e4806c3e0 msgr2=0x7f5e481f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.457+0000 7f5e4e400640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e4806c3e0 0x7f5e481f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.457+0000 7f5e4e400640 1 -- 172.21.15.40:0/2496962452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480b4820 con 0x7f5e481613b0 2024-03-08T06:07:00.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.457+0000 7f5e4e400640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e481613b0 0x7f5e480ad530 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c7d0 tx=0x7f5e4000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:07:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.458+0000 7f5e45600640 1 -- 172.21.15.40:0/2496962452 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40018070 con 0x7f5e481613b0 2024-03-08T06:07:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.458+0000 7f5e45600640 1 -- 172.21.15.40:0/2496962452 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5e4000a640 con 0x7f5e481613b0 2024-03-08T06:07:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.458+0000 7f5e45600640 1 -- 172.21.15.40:0/2496962452 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000aa90 con 0x7f5e481613b0 2024-03-08T06:07:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.458+0000 7f5e50400640 1 -- 172.21.15.40:0/2496962452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e480b4b00 con 0x7f5e481613b0 2024-03-08T06:07:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.458+0000 7f5e50400640 1 -- 172.21.15.40:0/2496962452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e480b50c0 con 0x7f5e481613b0 2024-03-08T06:07:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.459+0000 7f5e50400640 1 -- 172.21.15.40:0/2496962452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e480683c0 con 0x7f5e481613b0 2024-03-08T06:07:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.460+0000 7f5e45600640 1 -- 172.21.15.40:0/2496962452 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e4001b070 con 0x7f5e481613b0 2024-03-08T06:07:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.461+0000 7f5e45600640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2806b4d0 0x7f5e2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T06:07:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.461+0000 7f5e45600640 1 -- 172.21.15.40:0/2496962452 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5e40089570 con 0x7f5e481613b0 2024-03-08T06:07:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.461+0000 7f5e4da00640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2806b4d0 0x7f5e2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T06:07:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.462+0000 7f5e4da00640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2806b4d0 0x7f5e2806d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c036000 tx=0x7f5e3c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T06:07:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.465+0000 7f5e45600640 1 -- 172.21.15.40:0/2496962452 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e400586e0 con 0x7f5e481613b0 2024-03-08T06:07:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.691+0000 7f5e50400640 1 -- 172.21.15.40:0/2496962452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e480cdf50 con 0x7f5e481613b0 2024-03-08T06:07:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.692+0000 7f5e45600640 1 -- 172.21.15.40:0/2496962452 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f5e40057fe0 con 0x7f5e481613b0 2024-03-08T06:07:00.693 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-03-08T06:07:00.693 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/2194146146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6835","nonce":2194146146},{"type":"v1","addr":"172.21.15.40:6837","nonce":2194146146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/831964898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":831964898},{"type":"v1","addr":"172.21.15.53:6835","nonce":831964898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4723,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2951334634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2951334634},{"type":"v1","addr":"172.21.15.53:6833","nonce":2951334634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5412,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6836/2488929719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2488929719},{"type":"v1","addr":"172.21.15.40:6836","nonce":2488929719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[]} 2024-03-08T06:07:00.694 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 18 2024-03-08T06:07:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.694+0000 7f5e50400640 1 -- 172.21.15.40:0/2496962452 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2806b4d0 msgr2=0x7f5e2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.694+0000 7f5e50400640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2806b4d0 0x7f5e2806d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c036000 tx=0x7f5e3c008030 comp rx=0 tx=0).stop 2024-03-08T06:07:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.694+0000 7f5e50400640 1 -- 172.21.15.40:0/2496962452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e481613b0 msgr2=0x7f5e480ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T06:07:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.694+0000 7f5e50400640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e481613b0 0x7f5e480ad530 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c7d0 tx=0x7f5e4000cc90 comp rx=0 tx=0).stop 2024-03-08T06:07:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.695+0000 7f5e50400640 1 -- 172.21.15.40:0/2496962452 shutdown_connections 2024-03-08T06:07:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.695+0000 7f5e50400640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.40:6832/448564530,v1:172.21.15.40:6833/448564530] conn(0x7f5e2806b4d0 0x7f5e2806d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.695+0000 7f5e50400640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e481613b0 0x7f5e480ad530 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.695+0000 7f5e50400640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5e480b9500 0x7f5e480acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.695+0000 7f5e50400640 1 --2- 172.21.15.40:0/2496962452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e4806c3e0 0x7f5e481f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T06:07:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.695+0000 7f5e50400640 1 -- 172.21.15.40:0/2496962452 >> 172.21.15.40:0/2496962452 conn(0x7f5e480729b0 msgr2=0x7f5e480c82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T06:07:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.695+0000 7f5e50400640 1 -- 172.21.15.40:0/2496962452 shutdown_connections 2024-03-08T06:07:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-03-08T06:07:00.695+0000 7f5e50400640 1 -- 172.21.15.40:0/2496962452 wait complete. 2024-03-08T06:07:00.714 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-08T06:07:00.714 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-08T06:07:00.714 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-08T06:07:00.714 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:00.732 INFO:tasks.ceph.mds.a:Stopped 2024-03-08T06:07:00.732 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-08T06:07:00.732 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:00.795 INFO:tasks.ceph.mds.c:Stopped 2024-03-08T06:07:00.795 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-08T06:07:00.795 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:00.816 INFO:tasks.ceph.mds.b:Stopped 2024-03-08T06:07:00.816 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-08T06:07:00.816 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:00.880 INFO:tasks.ceph.mds.d:Stopped 2024-03-08T06:07:00.880 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-08T06:07:00.880 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-08T06:07:00.881 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:00.941 INFO:tasks.ceph.osd.0:Stopped 2024-03-08T06:07:00.942 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-08T06:07:00.942 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.006 INFO:tasks.ceph.osd.1:Stopped 2024-03-08T06:07:01.007 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-08T06:07:01.007 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.062 INFO:tasks.ceph.osd.2:Stopped 2024-03-08T06:07:01.062 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-08T06:07:01.062 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.128 INFO:tasks.ceph.osd.3:Stopped 2024-03-08T06:07:01.128 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-08T06:07:01.128 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.175 INFO:tasks.ceph.osd.4:Stopped 2024-03-08T06:07:01.175 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-08T06:07:01.175 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.246 INFO:tasks.ceph.osd.5:Stopped 2024-03-08T06:07:01.246 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-08T06:07:01.246 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.295 INFO:tasks.ceph.osd.6:Stopped 2024-03-08T06:07:01.296 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-08T06:07:01.296 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.363 INFO:tasks.ceph.osd.7:Stopped 2024-03-08T06:07:01.363 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-08T06:07:01.363 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-08T06:07:01.364 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.449 INFO:tasks.ceph.mgr.y:Stopped 2024-03-08T06:07:01.449 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-08T06:07:01.449 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.517 INFO:tasks.ceph.mgr.x:Stopped 2024-03-08T06:07:01.517 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-08T06:07:01.517 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-08T06:07:01.517 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.547 INFO:tasks.ceph.mon.a:Stopped 2024-03-08T06:07:01.548 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-08T06:07:01.548 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.576 INFO:tasks.ceph.mon.b:Stopped 2024-03-08T06:07:01.576 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-08T06:07:01.576 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T06:07:01.642 INFO:tasks.ceph.mon.c:Stopped 2024-03-08T06:07:01.643 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-08T06:07:01.643 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'missing required features' | head -n 1 2024-03-08T06:07:01.758 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:07:01.758 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-08T06:07:02.050 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:07:02.050 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-08T06:07:02.226 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:07:02.227 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-08T06:07:02.405 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:07:02.406 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-08T06:07:02.612 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:07:02.612 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-08T06:07:02.917 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:07:02.917 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-08T06:07:03.061 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:07:03.062 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-08T06:07:03.191 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:07:03.191 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-08T06:07:03.404 INFO:tasks.ceph:Archiving mon data... 2024-03-08T06:07:03.405 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425/data/mon.a.tgz 2024-03-08T06:07:03.405 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-03-08T06:07:03.466 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.Uq1XVrPvEq 2024-03-08T06:07:03.466 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Uq1XVrPvEq 2024-03-08T06:07:03.656 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.Uq1XVrPvEq 2024-03-08T06:07:03.814 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.Uq1XVrPvEq is 293KB 2024-03-08T06:07:03.926 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.Uq1XVrPvEq 2024-03-08T06:07:03.984 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425/data/mon.b.tgz 2024-03-08T06:07:03.984 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-03-08T06:07:04.039 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.Vacrk8axI9 2024-03-08T06:07:04.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Vacrk8axI9 2024-03-08T06:07:04.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.Vacrk8axI9 2024-03-08T06:07:04.412 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.Vacrk8axI9 is 392KB 2024-03-08T06:07:04.484 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.Vacrk8axI9 2024-03-08T06:07:04.542 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425/data/mon.c.tgz 2024-03-08T06:07:04.542 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-03-08T06:07:04.596 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.AUQv8gXeI5 2024-03-08T06:07:04.596 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.AUQv8gXeI5 2024-03-08T06:07:04.783 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.AUQv8gXeI5 2024-03-08T06:07:04.938 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.AUQv8gXeI5 is 392KB 2024-03-08T06:07:05.048 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.AUQv8gXeI5 2024-03-08T06:07:05.105 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-08T06:07:05.105 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-08T06:07:05.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-08T06:07:05.270 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-08T06:07:05.329 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-08T06:07:05.329 INFO:tasks.ceph:Archiving crash dumps... 2024-03-08T06:07:05.331 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425/remote/smithi040/crash 2024-03-08T06:07:05.331 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-08T06:07:05.434 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425/remote/smithi053/crash 2024-03-08T06:07:05.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-08T06:07:05.539 INFO:tasks.ceph:Shutting down logrotate 2024-03-08T06:07:05.539 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:07:05.547 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T06:07:05.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-08T06:07:05.794 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-08T06:07:05.898 INFO:tasks.ceph:Compressing logs... 2024-03-08T06:07:05.898 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-08T06:07:05.907 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-08T06:07:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49342.log 2024-03-08T06:07:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51108.log 2024-03-08T06:07:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49737.log 2024-03-08T06:07:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51108.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.49342.log.gz 2024-03-08T06:07:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-08T06:07:06.001 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51108.log.gz 2024-03-08T06:07:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49737.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49737.log.gz 2024-03-08T06:07:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47516.log 2024-03-08T06:07:06.002 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-08T06:07:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48948.log 2024-03-08T06:07:06.004 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42942.log 2024-03-08T06:07:06.004 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47516.log.gz 2024-03-08T06:07:06.010 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43728.log 2024-03-08T06:07:06.010 INFO:teuthology.orchestra.run.smithi040.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.48948.log.gz 2024-03-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42942.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42942.log.gz 2024-03-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50209.log 2024-03-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50335.log 2024-03-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43728.log.gz 2024-03-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49695.log 2024-03-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50209.log.gz 2024-03-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36899.log 2024-03-08T06:07:06.012 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50335.log.gz 2024-03-08T06:07:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46109.log 2024-03-08T06:07:06.012 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49695.log.gz 2024-03-08T06:07:06.017 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42000.log 2024-03-08T06:07:06.017 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42579.log 2024-03-08T06:07:06.017 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.3.42000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42000.log.gz 2024-03-08T06:07:06.017 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37111.log 2024-03-08T06:07:06.017 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.1.42579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42579.log.gz 2024-03-08T06:07:06.017 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41613.log 2024-03-08T06:07:06.018 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37231.log 2024-03-08T06:07:06.018 INFO:teuthology.orchestra.run.smithi053.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37111.log.gz 2024-03-08T06:07:06.018 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.1.41613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42575.log 2024-03-08T06:07:06.018 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41613.log.gz 2024-03-08T06:07:06.018 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37031.log 2024-03-08T06:07:06.018 INFO:teuthology.orchestra.run.smithi053.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37231.log.gz 2024-03-08T06:07:06.018 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.1.42575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42575.log.gz 2024-03-08T06:07:06.019 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37191.log 2024-03-08T06:07:06.019 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37031.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-08T06:07:06.019 INFO:teuthology.orchestra.run.smithi053.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37031.log.gz 2024-03-08T06:07:06.019 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37191.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-08T06:07:06.019 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37191.log.gz 2024-03-08T06:07:06.019 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-08T06:07:06.019 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36951.log 2024-03-08T06:07:06.019 INFO:teuthology.orchestra.run.smithi053.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-08T06:07:06.020 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.4.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-08T06:07:06.020 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-08T06:07:06.022 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51570.log 2024-03-08T06:07:06.022 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36899.log.gz 2024-03-08T06:07:06.022 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2024-03-08T06:07:06.022 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46109.log.gz 2024-03-08T06:07:06.022 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51360.log 2024-03-08T06:07:06.023 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51570.log.gz 2024-03-08T06:07:06.023 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45026.log 2024-03-08T06:07:06.023 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gz 2024-03-08T06:07:06.023 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51444.log 2024-03-08T06:07:06.023 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51360.log.gz 2024-03-08T06:07:06.023 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.2.45026.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.2.45026.log.gz -5 --verbose 2024-03-08T06:07:06.023 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.44064.log 2024-03-08T06:07:06.024 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50167.log 2024-03-08T06:07:06.024 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51444.log.gz 2024-03-08T06:07:06.024 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45605.log 2024-03-08T06:07:06.024 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44064.log.gz 2024-03-08T06:07:06.024 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42720.log 2024-03-08T06:07:06.024 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50167.log.gz 2024-03-08T06:07:06.025 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45605.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45605.log.gz 2024-03-08T06:07:06.025 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48612.log 2024-03-08T06:07:06.025 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51150.log 2024-03-08T06:07:06.032 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36951.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-08T06:07:06.032 INFO:teuthology.orchestra.run.smithi053.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36951.log.gz 2024-03-08T06:07:06.032 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41591.log 2024-03-08T06:07:06.032 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36991.log 2024-03-08T06:07:06.033 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.1.41591.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-08T06:07:06.033 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41591.log.gz 2024-03-08T06:07:06.033 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.36991.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36991.log.gz 2024-03-08T06:07:06.033 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42004.log 2024-03-08T06:07:06.033 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-08T06:07:06.033 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.3.42004.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42004.log.gz 2024-03-08T06:07:06.033 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37071.log 2024-03-08T06:07:06.034 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37151.log 2024-03-08T06:07:06.034 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37071.log: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-08T06:07:06.034 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37071.log.gz 2024-03-08T06:07:06.034 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-08T06:07:06.034 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.37151.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37151.log.gz 2024-03-08T06:07:06.034 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-08T06:07:06.034 INFO:teuthology.orchestra.run.smithi053.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-08T06:07:06.035 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42720.log.gz 2024-03-08T06:07:06.035 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43264.log 2024-03-08T06:07:06.035 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48612.log.gz 2024-03-08T06:07:06.035 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42151.log 2024-03-08T06:07:06.035 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51150.log.gz 2024-03-08T06:07:06.035 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45731.log 2024-03-08T06:07:06.035 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43264.log.gz 2024-03-08T06:07:06.036 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46634.log 2024-03-08T06:07:06.036 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42151.log.gz 2024-03-08T06:07:06.036 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50982.log 2024-03-08T06:07:06.036 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45731.log.gz 2024-03-08T06:07:06.036 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51024.log 2024-03-08T06:07:06.036 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46634.log.gz 2024-03-08T06:07:06.036 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50025.log 2024-03-08T06:07:06.037 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50982.log.gz 2024-03-08T06:07:06.037 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51486.log 2024-03-08T06:07:06.037 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51024.log.gz 2024-03-08T06:07:06.037 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46151.log 2024-03-08T06:07:06.038 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-08T06:07:06.039 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 91.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-08T06:07:06.042 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50025.log.gz 2024-03-08T06:07:06.042 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47660.log 2024-03-08T06:07:06.042 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51486.log.gz 2024-03-08T06:07:06.042 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43276.log 2024-03-08T06:07:06.043 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46151.log.gz 2024-03-08T06:07:06.043 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44722.log 2024-03-08T06:07:06.043 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47660.log.gz 2024-03-08T06:07:06.043 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47966.log 2024-03-08T06:07:06.043 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43276.log.gz 2024-03-08T06:07:06.043 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44722.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44722.log.gz 2024-03-08T06:07:06.043 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47348.log 2024-03-08T06:07:06.044 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47805.log 2024-03-08T06:07:06.044 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47966.log.gz 2024-03-08T06:07:06.044 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50732.log 2024-03-08T06:07:06.044 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47348.log.gz 2024-03-08T06:07:06.044 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47805.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30840.log 2024-03-08T06:07:06.044 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50732.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47805.log.gz 2024-03-08T06:07:06.044 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50732.log.gz 2024-03-08T06:07:06.045 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48177.log 2024-03-08T06:07:06.045 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.tmp-client.admin.30840.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30840.log.gzgzip 2024-03-08T06:07:06.045 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42944.log 2024-03-08T06:07:06.047 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43272.log 2024-03-08T06:07:06.047 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48177.log.gz 2024-03-08T06:07:06.047 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47054.log 2024-03-08T06:07:06.048 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42944.log.gz 2024-03-08T06:07:06.048 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51738.log 2024-03-08T06:07:06.048 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43272.log.gz 2024-03-08T06:07:06.048 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47054.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47054.log.gz 2024-03-08T06:07:06.048 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50377.log 2024-03-08T06:07:06.048 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49857.log 2024-03-08T06:07:06.048 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51738.log.gz 2024-03-08T06:07:06.050 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43644.log 2024-03-08T06:07:06.050 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50377.log.gz 2024-03-08T06:07:06.050 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47180.log 2024-03-08T06:07:06.050 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49857.log.gz 2024-03-08T06:07:06.050 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43644.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43644.log.gz 2024-03-08T06:07:06.050 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49205.log 2024-03-08T06:07:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42943.log 2024-03-08T06:07:06.052 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47180.log.gz 2024-03-08T06:07:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.49205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49205.log.gz 2024-03-08T06:07:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51654.log 2024-03-08T06:07:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43560.log 2024-03-08T06:07:06.053 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42943.log.gz 2024-03-08T06:07:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51654.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51654.log.gz 2024-03-08T06:07:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46388.log 2024-03-08T06:07:06.055 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-03-08T06:07:06.055 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43560.log.gz 2024-03-08T06:07:06.056 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45941.log 2024-03-08T06:07:06.056 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46388.log.gz 2024-03-08T06:07:06.057 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49899.log 2024-03-08T06:07:06.057 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2024-03-08T06:07:06.058 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45941.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45941.log.gz -5 2024-03-08T06:07:06.058 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50606.log 2024-03-08T06:07:06.059 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48051.log 2024-03-08T06:07:06.059 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49899.log.gz 2024-03-08T06:07:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48570.log 2024-03-08T06:07:06.060 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50606.log.gz 2024-03-08T06:07:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45899.log 2024-03-08T06:07:06.061 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48051.log.gz 2024-03-08T06:07:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50461.log 2024-03-08T06:07:06.062 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48570.log.gz 2024-03-08T06:07:06.067 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45899.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-08T06:07:06.067 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45899.log.gz 2024-03-08T06:07:06.068 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46304.log 2024-03-08T06:07:06.068 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50461.log.gz 2024-03-08T06:07:06.068 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51841.log 2024-03-08T06:07:06.068 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46304.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46304.log.gz 2024-03-08T06:07:06.068 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42926.log 2024-03-08T06:07:06.068 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-08T06:07:06.068 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51841.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51841.log.gz 2024-03-08T06:07:06.068 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44574.log 2024-03-08T06:07:06.070 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45773.log 2024-03-08T06:07:06.070 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42926.log.gz 2024-03-08T06:07:06.070 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.44574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44574.log.gz 2024-03-08T06:07:06.070 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48261.log 2024-03-08T06:07:06.071 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49653.log 2024-03-08T06:07:06.071 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45773.log.gz 2024-03-08T06:07:06.072 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50554.log 2024-03-08T06:07:06.072 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48261.log.gz 2024-03-08T06:07:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45857.log 2024-03-08T06:07:06.073 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49653.log.gz 2024-03-08T06:07:06.074 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45479.log 2024-03-08T06:07:06.074 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50554.log.gz 2024-03-08T06:07:06.075 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46676.log 2024-03-08T06:07:06.075 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45857.log.gz 2024-03-08T06:07:06.076 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48008.log 2024-03-08T06:07:06.076 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45479.log.gz 2024-03-08T06:07:06.077 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46676.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46676.log.gz 2024-03-08T06:07:06.077 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51612.log 2024-03-08T06:07:06.077 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43896.log 2024-03-08T06:07:06.078 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48008.log.gz 2024-03-08T06:07:06.078 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51192.log 2024-03-08T06:07:06.079 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51612.log.gz 2024-03-08T06:07:06.079 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51696.log 2024-03-08T06:07:06.080 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43896.log.gz 2024-03-08T06:07:06.080 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44022.log 2024-03-08T06:07:06.081 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51192.log.gz 2024-03-08T06:07:06.081 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45647.log 2024-03-08T06:07:06.082 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51696.log.gz 2024-03-08T06:07:06.082 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43602.log 2024-03-08T06:07:06.082 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44022.log.gz 2024-03-08T06:07:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46934.log 2024-03-08T06:07:06.084 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45647.log.gz 2024-03-08T06:07:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43602.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43602.log.gz 2024-03-08T06:07:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41838.log 2024-03-08T06:07:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50251.log 2024-03-08T06:07:06.085 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46934.log.gz 2024-03-08T06:07:06.086 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48475.log 2024-03-08T06:07:06.086 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41838.log.gz 2024-03-08T06:07:06.087 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43770.log 2024-03-08T06:07:06.088 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50251.log.gz 2024-03-08T06:07:06.088 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48135.log 2024-03-08T06:07:06.088 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48475.log.gz 2024-03-08T06:07:06.089 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45311.log 2024-03-08T06:07:06.090 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43770.log.gz 2024-03-08T06:07:06.090 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37283.log 2024-03-08T06:07:06.090 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48135.log.gz 2024-03-08T06:07:06.091 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47390.log 2024-03-08T06:07:06.092 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45311.log.gz 2024-03-08T06:07:06.092 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37283.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37283.log.gz 2024-03-08T06:07:06.092 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50419.log 2024-03-08T06:07:06.093 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42289.log 2024-03-08T06:07:06.094 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47390.log.gz 2024-03-08T06:07:06.094 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50419.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50419.log.gz 2024-03-08T06:07:06.094 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37156.log 2024-03-08T06:07:06.095 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42939.log 2024-03-08T06:07:06.096 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42289.log.gz 2024-03-08T06:07:06.096 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51066.log 2024-03-08T06:07:06.097 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37156.log.gz 2024-03-08T06:07:06.097 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42937.log 2024-03-08T06:07:06.098 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42939.log.gz 2024-03-08T06:07:06.098 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51066.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51066.log.gz 2024-03-08T06:07:06.099 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51402.log 2024-03-08T06:07:06.100 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49474.log 2024-03-08T06:07:06.100 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42937.log.gz 2024-03-08T06:07:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51402.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-08T06:07:06.101 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51402.log.gz 2024-03-08T06:07:06.102 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50503.log 2024-03-08T06:07:06.102 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.1.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49474.log.gz 2024-03-08T06:07:06.103 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49282.log 2024-03-08T06:07:06.104 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43854.log 2024-03-08T06:07:06.104 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50503.log.gz 2024-03-08T06:07:06.105 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49282.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49282.log.gz 2024-03-08T06:07:06.105 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43938.log 2024-03-08T06:07:06.106 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42045.log 2024-03-08T06:07:06.106 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43854.log.gz 2024-03-08T06:07:06.107 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48528.log 2024-03-08T06:07:06.107 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43938.log.gz 2024-03-08T06:07:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42045.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42045.log.gz 2024-03-08T06:07:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46766.log 2024-03-08T06:07:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42340.log 2024-03-08T06:07:06.110 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48528.log.gz 2024-03-08T06:07:06.111 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42594.log 2024-03-08T06:07:06.111 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46766.log.gz 2024-03-08T06:07:06.112 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42340.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42340.log.gz 2024-03-08T06:07:06.112 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43273.log 2024-03-08T06:07:06.113 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51528.log 2024-03-08T06:07:06.113 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42594.log.gz 2024-03-08T06:07:06.114 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49983.log 2024-03-08T06:07:06.114 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43273.log.gz 2024-03-08T06:07:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51528.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.51528.log.gz --verbose 2024-03-08T06:07:06.115 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.0.44593.log 2024-03-08T06:07:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46508.log 2024-03-08T06:07:06.116 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49983.log.gz 2024-03-08T06:07:06.117 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.44593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43270.log 2024-03-08T06:07:06.117 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44593.log.gz 2024-03-08T06:07:06.118 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45022.log 2024-03-08T06:07:06.118 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46508.log.gz 2024-03-08T06:07:06.119 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43270.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43270.log.gz 2024-03-08T06:07:06.119 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48906.log 2024-03-08T06:07:06.120 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.2.45022.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45022.log.gz 2024-03-08T06:07:06.121 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47264.log 2024-03-08T06:07:06.122 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45815.log 2024-03-08T06:07:06.122 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48906.log.gz 2024-03-08T06:07:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47763.log 2024-03-08T06:07:06.123 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47264.log.gz 2024-03-08T06:07:06.124 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45815.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-08T06:07:06.124 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45815.log.gz 2024-03-08T06:07:06.125 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51318.log 2024-03-08T06:07:06.125 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47763.log.gz 2024-03-08T06:07:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41978.log 2024-03-08T06:07:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51318.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51318.log.gz 2024-03-08T06:07:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46262.log 2024-03-08T06:07:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41978.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41978.log.gz 2024-03-08T06:07:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48654.log 2024-03-08T06:07:06.130 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46808.log 2024-03-08T06:07:06.131 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46262.log.gz 2024-03-08T06:07:06.131 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48654.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48654.log.gz 2024-03-08T06:07:06.131 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42245.log 2024-03-08T06:07:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49941.log 2024-03-08T06:07:06.133 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46808.log.gz 2024-03-08T06:07:06.134 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51780.log 2024-03-08T06:07:06.134 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42245.log.gz 2024-03-08T06:07:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46592.log 2024-03-08T06:07:06.135 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49941.log.gz 2024-03-08T06:07:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42100.log 2024-03-08T06:07:06.137 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51780.log.gz 2024-03-08T06:07:06.138 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42934.log 2024-03-08T06:07:06.138 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46592.log.gz 2024-03-08T06:07:06.139 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45353.log 88.7% 2024-03-08T06:07:06.139 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42100.log.gz 2024-03-08T06:07:06.140 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43518.log 2024-03-08T06:07:06.141 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42934.log.gz 2024-03-08T06:07:06.141 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50125.log 2024-03-08T06:07:06.141 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45353.log.gz 2024-03-08T06:07:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43518.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43518.log.gz 2024-03-08T06:07:06.143 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51933.log 2024-03-08T06:07:06.143 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47474.log 2024-03-08T06:07:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51933.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50125.log.gz 2024-03-08T06:07:06.145 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose 87.8% -- /var/log/ceph/ceph-client.admin.48864.log -- replaced with /var/log/ceph/ceph-client.admin.51933.log.gz 2024-03-08T06:07:06.145 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-03-08T06:07:06.146 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49611.log 2024-03-08T06:07:06.146 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47474.log.gz 2024-03-08T06:07:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48864.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.48864.log.gz 2024-03-08T06:07:06.148 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41742.log 2024-03-08T06:07:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45521.log 2024-03-08T06:07:06.149 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49611.log.gz 2024-03-08T06:07:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41742.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41742.log.gz 2024-03-08T06:07:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46025.log 2024-03-08T06:07:06.151 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43686.log/var/log/ceph/ceph-client.admin.45521.log: 2024-03-08T06:07:06.151 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45521.log.gz 2024-03-08T06:07:06.152 INFO:teuthology.orchestra.run.smithi053.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-08T06:07:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46025.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46025.log.gz 2024-03-08T06:07:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47138.log 2024-03-08T06:07:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47012.log 2024-03-08T06:07:06.154 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43686.log.gz 2024-03-08T06:07:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47138.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47138.log.gz 2024-03-08T06:07:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51234.log 2024-03-08T06:07:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47885.log 2024-03-08T06:07:06.156 INFO:teuthology.orchestra.run.smithi053.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-08T06:07:06.157 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47012.log.gz 2024-03-08T06:07:06.157 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47096.log 2024-03-08T06:07:06.157 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51234.log.gz 2024-03-08T06:07:06.158 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46550.log 2024-03-08T06:07:06.158 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47885.log.gz 2024-03-08T06:07:06.159 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47096.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47096.log.gz 2024-03-08T06:07:06.159 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48387.log 2024-03-08T06:07:06.160 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48738.log 2024-03-08T06:07:06.161 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46550.log.gz 2024-03-08T06:07:06.161 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48387.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48387.log.gz 2024-03-08T06:07:06.162 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45689.log 2024-03-08T06:07:06.162 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48738.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-08T06:07:06.163 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48738.log.gz 2024-03-08T06:07:06.164 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45689.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45689.log.gz 2024-03-08T06:07:06.164 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43980.log 2024-03-08T06:07:06.165 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42468.log 2024-03-08T06:07:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43980.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43980.log.gz 2024-03-08T06:07:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47618.log 2024-03-08T06:07:06.167 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45216.log 2024-03-08T06:07:06.168 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42468.log.gz 2024-03-08T06:07:06.168 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47618.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47618.log.gz 2024-03-08T06:07:06.168 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49014.log 2024-03-08T06:07:06.169 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42552.log 2024-03-08T06:07:06.170 INFO:teuthology.orchestra.run.smithi053.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-08T06:07:06.170 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45216.log.gz 2024-03-08T06:07:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49014.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49014.log.gz 2024-03-08T06:07:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45983.log 2024-03-08T06:07:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49201.log 2024-03-08T06:07:06.172 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42552.log.gz 2024-03-08T06:07:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46430.log 2024-03-08T06:07:06.173 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45983.log.gz 2024-03-08T06:07:06.174 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.49201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42382.log 2024-03-08T06:07:06.174 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49201.log.gz 2024-03-08T06:07:06.175 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46430.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.46430.log.gz 2024-03-08T06:07:06.175 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47432.log 2024-03-08T06:07:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48345.log 2024-03-08T06:07:06.177 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42382.log.gz 2024-03-08T06:07:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47432.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47432.log.gz 2024-03-08T06:07:06.178 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48431.log 2024-03-08T06:07:06.178 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43476.log 2024-03-08T06:07:06.179 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48431.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48345.log.gz 2024-03-08T06:07:06.180 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48431.log.gz 2024-03-08T06:07:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-08T06:07:06.181 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51889.log 2024-03-08T06:07:06.182 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43476.log.gz 2024-03-08T06:07:06.182 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44244.log 2024-03-08T06:07:06.184 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42932.log 2024-03-08T06:07:06.184 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51889.log.gz 2024-03-08T06:07:06.185 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44244.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44244.log.gz 2024-03-08T06:07:06.186 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42932.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42932.log.gz 2024-03-08T06:07:06.187 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50293.log 2024-03-08T06:07:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42678.log 2024-03-08T06:07:06.189 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48780.log 2024-03-08T06:07:06.190 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50293.log.gz 2024-03-08T06:07:06.191 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42678.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42678.log.gz 2024-03-08T06:07:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48780.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48780.log.gz 2024-03-08T06:07:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46346.log 2024-03-08T06:07:06.194 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45155.log 2024-03-08T06:07:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43280.log 2024-03-08T06:07:06.195 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46346.log.gz 2024-03-08T06:07:06.197 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45155.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45155.log.gz 2024-03-08T06:07:06.198 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43280.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43280.log.gz 2024-03-08T06:07:06.198 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-08T06:07:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45563.log 2024-03-08T06:07:06.201 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41919.log 2024-03-08T06:07:06.202 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45563.log: 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-08T06:07:06.202 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45563.log.gz 2024-03-08T06:07:06.203 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41919.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41919.log.gz 2024-03-08T06:07:06.204 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-08T06:07:06.205 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43812.log 2024-03-08T06:07:06.207 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45437.log 2024-03-08T06:07:06.208 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43812.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43812.log.gz 2024-03-08T06:07:06.209 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45437.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-08T06:07:06.210 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45437.log.gz 2024-03-08T06:07:06.210 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47718.log 2024-03-08T06:07:06.213 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.47718.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47718.log.gz 2024-03-08T06:07:06.213 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.46067.log 2024-03-08T06:07:06.214 INFO:teuthology.orchestra.run.smithi040.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-08T06:07:06.215 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46067.log: -- /var/log/ceph/ceph-client.admin.50083.log 2024-03-08T06:07:06.216 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46067.log.gz 2024-03-08T06:07:06.217 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48696.log 2024-03-08T06:07:06.218 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50083.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50083.log.gz 2024-03-08T06:07:06.219 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42510.log 2024-03-08T06:07:06.220 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48696.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48696.log.gz 2024-03-08T06:07:06.222 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42510.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42510.log.gz 2024-03-08T06:07:06.222 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42636.log 2024-03-08T06:07:06.224 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-08T06:07:06.224 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.51276.log 2024-03-08T06:07:06.225 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42636.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42636.log.gz 2024-03-08T06:07:06.226 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48219.log 2024-03-08T06:07:06.227 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.51276.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51276.log.gz 2024-03-08T06:07:06.227 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50940.log 2024-03-08T06:07:06.229 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48219.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48219.log.gz 2024-03-08T06:07:06.229 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50690.log 2024-03-08T06:07:06.230 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43268.log 2024-03-08T06:07:06.231 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50940.log.gz 2024-03-08T06:07:06.232 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50690.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50690.log.gz -5 2024-03-08T06:07:06.232 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45269.log 2024-03-08T06:07:06.233 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43268.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42762.log 2024-03-08T06:07:06.233 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43268.log.gz 2024-03-08T06:07:06.234 INFO:teuthology.orchestra.run.smithi053.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-08T06:07:06.234 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49779.log 2024-03-08T06:07:06.235 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45269.log.gz 2024-03-08T06:07:06.236 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48093.log 2024-03-08T06:07:06.236 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42762.log.gz 2024-03-08T06:07:06.237 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.49779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47306.log 2024-03-08T06:07:06.237 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49779.log.gz 2024-03-08T06:07:06.239 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48093.log: gzip 88.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.48093.log.gz -- 2024-03-08T06:07:06.239 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.46892.log 2024-03-08T06:07:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47222.log 2024-03-08T06:07:06.240 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47306.log.gz 2024-03-08T06:07:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44106.log 2024-03-08T06:07:06.241 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46892.log.gz 2024-03-08T06:07:06.242 INFO:teuthology.orchestra.run.smithi040.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-08T06:07:06.243 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47576.log 2024-03-08T06:07:06.243 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47222.log.gz 2024-03-08T06:07:06.243 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48303.log 2024-03-08T06:07:06.244 INFO:teuthology.orchestra.run.smithi053.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-08T06:07:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47576.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.44106.log.gz 2024-03-08T06:07:06.245 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45395.log 2024-03-08T06:07:06.245 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47576.log.gz 2024-03-08T06:07:06.246 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48303.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48303.log.gz 2024-03-08T06:07:06.246 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50648.log 2024-03-08T06:07:06.247 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46850.log 2024-03-08T06:07:06.247 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45395.log.gz 2024-03-08T06:07:06.248 INFO:teuthology.orchestra.run.smithi053.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-08T06:07:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.50648.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50648.log.gzgzip 2024-03-08T06:07:06.249 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-08T06:07:06.250 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48822.log 2024-03-08T06:07:06.251 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46850.log.gz 2024-03-08T06:07:06.252 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42426.log 2024-03-08T06:07:06.253 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.48822.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48822.log.gz 2024-03-08T06:07:06.253 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-03-08T06:07:06.253 INFO:teuthology.orchestra.run.smithi053.stderr:real 0m0.294s 2024-03-08T06:07:06.254 INFO:teuthology.orchestra.run.smithi053.stderr:user 0m1.026s 2024-03-08T06:07:06.254 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m0.222s 2024-03-08T06:07:06.255 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42426.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42426.log.gz 2024-03-08T06:07:06.350 INFO:teuthology.orchestra.run.smithi040.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-08T06:07:06.389 INFO:teuthology.orchestra.run.smithi040.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-08T06:07:06.418 INFO:teuthology.orchestra.run.smithi040.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-08T06:07:06.621 INFO:teuthology.orchestra.run.smithi040.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-08T06:07:07.467 INFO:teuthology.orchestra.run.smithi040.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-08T06:07:07.471 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-03-08T06:07:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m1.517s 2024-03-08T06:07:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:user 0m2.556s 2024-03-08T06:07:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m0.805s 2024-03-08T06:07:07.473 INFO:tasks.ceph:Archiving logs... 2024-03-08T06:07:07.474 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425/remote/smithi040/log 2024-03-08T06:07:07.475 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-08T06:07:08.071 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425/remote/smithi053/log 2024-03-08T06:07:08.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-08T06:07:08.315 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-08T06:07:08.325 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-08T06:07:08.326 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-08T06:07:08.334 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-08T06:07:08.494 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-08T06:07:08.503 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-08T06:07:08.504 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T06:07:08.512 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T06:07:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-03-08T06:07:08.569 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-03-08T06:07:08.572 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T06:07:08.572 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-03-08T06:07:08.572 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 +1697us[+1697us] +/- 112ms 2024-03-08T06:07:08.572 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -1428us[-1426us] +/- 72ms 2024-03-08T06:07:08.572 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv03.front.sepia.ceph.com 3 6 377 31 +355us[ +358us] +/- 69ms 2024-03-08T06:07:08.572 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:07:08.580 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T06:07:08.580 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-03-08T06:07:08.580 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 31 +1716us[+1720us] +/- 112ms 2024-03-08T06:07:08.580 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -1411us[-1408us] +/- 72ms 2024-03-08T06:07:08.580 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv03.front.sepia.ceph.com 3 6 377 31 +365us[ +368us] +/- 69ms 2024-03-08T06:07:08.580 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T06:07:08.581 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-08T06:07:08.590 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-08T06:07:08.591 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-08T06:07:08.635 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-08T06:07:08.870 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-08T06:07:09.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T06:07:09.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:07:09.198 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T06:07:09.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:07:09.306 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-08T06:07:09.315 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-08T06:07:09.378 INFO:teuthology.task.internal:Duration was 796.496093 seconds 2024-03-08T06:07:09.378 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-08T06:07:09.428 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-08T06:07:09.428 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-08T06:07:09.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-08T06:07:09.608 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T06:07:09.609 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T06:07:10.139 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-08T06:07:10.139 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-03-08T06:07:10.140 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-08T06:07:10.212 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-03-08T06:07:10.212 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-08T06:07:10.289 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-08T06:07:10.289 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-08T06:07:10.297 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-08T06:07:10.395 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-08T06:07:10.395 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-08T06:07:10.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-08T06:07:10.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-08T06:07:10.600 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-08T06:07:10.600 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-08T06:07:10.700 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-08T06:07:10.804 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-08T06:07:10.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-08T06:07:10.822 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-08T06:07:10.927 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-03-08T06:07:10.931 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-03-08T06:07:10.984 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-08T06:07:11.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:07:11.037 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-08T06:07:11.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T06:07:11.096 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-08T06:07:11.106 INFO:teuthology.task.internal:Transferring archived files... 2024-03-08T06:07:11.107 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425/remote/smithi040 2024-03-08T06:07:11.107 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-08T06:07:11.220 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425/remote/smithi053 2024-03-08T06:07:11.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-08T06:07:11.336 INFO:teuthology.task.internal:Removing archive directory... 2024-03-08T06:07:11.336 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-08T06:07:11.344 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-08T06:07:11.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-08T06:07:11.416 INFO:teuthology.task.internal:Not uploading archives. 2024-03-08T06:07:11.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-08T06:07:11.447 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-08T06:07:11.447 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-08T06:07:11.455 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-08T06:07:11.507 INFO:teuthology.orchestra.run.smithi040.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 8 06:07 /home/ubuntu/cephtest 2024-03-08T06:07:11.516 INFO:teuthology.orchestra.run.smithi053.stdout: 395887 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 8 06:07 /home/ubuntu/cephtest 2024-03-08T06:07:11.520 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-08T06:07:11.531 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-08T06:07:11.622 INFO:teuthology.nuke:Checking targets against current locks 2024-03-08T06:07:11.648 DEBUG:teuthology.nuke:shortname: smithi040 2024-03-08T06:07:11.648 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T06:07:11.676 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 05:44:38.759492', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE31JNxLJ5xIvqhIrwCXCY/HPljTsB8D0lzrE06ebM6bqC9FPeEn+hA8Jfmm1hUfxwY8fvklHKl+ayl8QGGNwVM='} 2024-03-08T06:07:11.677 DEBUG:teuthology.nuke:shortname: smithi053 2024-03-08T06:07:11.677 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T06:07:11.701 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_05:43:46-fs:functional-main-distro-default-smithi/7585425', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 05:44:38.761226', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgUVmyKe/dO23KvZNq7LUj18qPW9uMLNPo7GkfiWAQj+cwmNKxO+iImtt3ST8tLKKwIeAz0qcdYZJTcyo+gcoQ='} 2024-03-08T06:07:11.732 INFO:teuthology.orchestra.console.smithi040:Power off 2024-03-08T06:07:11.732 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-08T06:07:11.759 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off 2024-03-08T06:07:11.759 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:07:11.776 INFO:teuthology.orchestra.console.smithi053:Power off 2024-03-08T06:07:11.776 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-08T06:07:11.786 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-03-08T06:07:11.800 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2024-03-08T06:07:11.800 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:07:11.822 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-03-08T06:07:15.788 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:07:15.823 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:07:15.899 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-03-08T06:07:15.933 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-03-08T06:07:19.902 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:07:19.934 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T06:07:20.015 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2024-03-08T06:07:20.045 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2024-03-08T06:07:20.116 INFO:teuthology.orchestra.console.smithi040:Power off completed 2024-03-08T06:07:20.147 INFO:teuthology.orchestra.console.smithi053:Power off completed 2024-03-08T06:07:20.253 INFO:teuthology.run:Summary data: client.0-kernel-sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 client.1-kernel-sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin} duration: 796.4960925579071 failure_reason: 'Test failure: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters)' flavor: default owner: scheduled_jcollin@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b4a7ad8048164354b4cde0de50a361ac status: fail success: false 2024-03-08T06:07:20.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-08T06:07:20.336 INFO:teuthology.run:FAIL